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Abstract — It is proved in this work that exhaustively de- 
termining bad patterns in arbitrary, finite low-density parity- 
check (LDPC) codes, including stopping sets for binary era- 
sure channels (BECs) and trapping sets (also known as near- 
codewords) for general memoryless symmetric channels, is an 
NP-complete problem, and efficient algorithms are provided for 
codes of practical short lengths n w 500. By exploiting the sparse 
connectivity of LDPC codes, the stopping sets of size < 13 and 
the trapping sets of size < 11 can be efficiently exhaustively 
determined for the first time, and the resulting exhaustive list is 
of great importance for code analysis and finite code optimization. 
The featured tree-based narrowing search distinguishes this 
algorithm from existing ones for which inexhaustive methods 
are employed. One important byproduct is a pair of upper 
bounds on the bit-error rate (BER) & frame-error rate (FER) 
iterative decoding performance of arbitrary codes over BECs 
that can be evaluated for any value of the erasure probability, 
including both the waterfall and the error floor regions. The 
tightness of these upper bounds and the exhaustion capability of 
the proposed algorithm are proved when combining an optimal 
leaf-finding module with the tree-based search. These upper 
bounds also provide a worst-case-performance guarantee which 
is crucial to optimizing LDPC codes for extremely low error 
rate applications, e.g., optical/satellite communications. Extensive 
numerical experiments are conducted that include both randomly 
and algebraically constructed LDPC codes, the results of which 
demonstrate the superior efficiency of the exhaustion algorithm 
and its significant value for finite length code optimization. 



I. Introduction 

Assuming iterative decoding [1], the error floor performance 
of arbitrary, fixed, finite low-density parity-check (LDPC) 
codes [2], [3] is dominated by the bad sub-structures resid- 
ing in the code of interest, which are termed stopping sets 
when binary erasure channels (BECs) are considered [4]. For 
transmitting over BECs, the stopping sets not only determine 
the error floor performance but also determine the entire bit- 
error rate (BER) and the frame-error rate (FER) curves of 
any LDPC code [4], [5]. For general memoryless binary- 
input/symmetric-output channels, the dominating error-prone 
patterns have different names motivated by different analytical 
techniques; these include trapping sets [6], near-codewords [7], 
pseudo-codewords [8], and instantons [9]. In this work, we 
consider the problem of exhaustively finding the stopping sets 
and trapping sets (near-codewords) of any fixed finite binary 
linear code. 
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Due to the prohibitive cost of computing the entire stopping 
set distribution [10] and the even greater expense of exhausting 
all trapping sets, in practice, the waterfall threshold of the 
BER is generally approximated using the density evolution 
[11], [12] with the corresponding scaling laws [13] for codes 
of different finite lengths, and pinpointed by Monte-Carlo 
simulation. On the other hand, the error floor can be lower 
bounded by identifying most of the dominant stopping sets 
[14], [6], by importance sampling [15], or by the instanton 
analysis [9], [16]. For the simplest non-trivial setting of con- 
sidering the random ensemble of finite codes over BECs, many 
insightful results have been obtained including the averaged 
performance [4], the corresponding waterfall scaling law [13], 
and the ensemble error floor analysis [17], [18]. Other research 
related to the stopping set analysis includes, for example, 
results on the asymptotic stopping set weight spectrum [19], 
the Hamming distance spectrum [20], [21], and the stopping 
redundancy [22]. 

The minimal stopping distance is defined as the minimal 
size of all non-empty stopping sets. As the minimal Hamming 
distance is to the maximum a posteriori probability (MAP) 
detector, the minimal stopping distance is the determining 
factor of code performance of iterative decoders over BECs in 
the low error rate regimes. Recently, determining the minimal 
stopping distance for an arbitrary binary linear code has been 
proven to be an NP-complete problem [23] which shows that 
there is little chance that a deterministic algorithm can be 
devised with polynomial computational complexity Poly(rt) 
with respect to the codeword length n. Although not explicitly 
stated in [23], it follows that the task of determining the 
minimal stopping distance for LDPC codes with sparse parity 
matrices is still NP-complete regardless of the sparsity con- 
straint on the codes of interest. By noting that for large LDPC 
codes, most small stopping sets are also valid codewords due 
to the combinatorial bias of random construction [18], this NP- 
completeness result is not totally unexpected since deciding 
the minimal Hamming distance of any linear code is one 
of the classic NP-complete problems of information theory 
[24], [25], [26]. For codes having special structure such as the 
Hamming codes, the LDPC codes based on circulant matrices, 
the convolutional LDPC codes etc., their minimal stopping / 
Hamming distances can be analytically determined through 
combinatorial or algebraic analyses [27], [28], [29], [30]. For 
general randomly constructed LDPC codes, no such result 
exists. Since the minimal stopping (or Hamming) distance is 
defined as the minimal size of all non-empty stopping sets 
(or codewords), upper bounds on the minimal stopping (or 
Hamming) distance can be obtained by explicitly identifying 
any single stopping set (or codeword) of small size, which is 
generally achieved by the error impulse methods [31], [32]. 

For non-erasure channels, the minimal trapping distance can 
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be defined analogously as the minimal size of all non-empty 
trapping sets. In Section III-B, it is proved that the problem of 
deciding the minimal trapping distance is also NP-complete. 
In other words, determining the asymptotic performance of 
the iterative detector is not an easier task when compared 
to determining the asymptotic performance of the optimal 
maximum a posteriori probability detector. The NP-hardness 
of this problem is also one of the reasons that all existing finite- 
code error-floor optimization schemes employ approximations 
or guided heuristics as the objective functions, such as the 
girth of the Tanner graph [33], [34], the Approximate Cycle 
Extrinsic (ACE) message degree [35], partial stopping set 
elimination [36], and ensemble-inspired upper bounds [37]. 
With already significant success based on the indirect metrics, 
one would expect greater improvement if the minimal stopping 
/ trapping distance can be directly employed. One purpose 
of this paper is to serve as a starting foundation for future 
research on efficient exhaustion algorithms for error-prone 
patterns, which can be used to decide the minimal stopping / 
trapping distance for corresponding code optimization. 

One major motivation for this work springs from the fact 
that the inherent NP-hardness of the minimal stopping / 
trapping distance problems only describes the asymptotic com- 
plexity with large codeword length n, and does not preclude 
efficient algorithms for codes of small lengths, say n s=s 500, 
which is of great importance in practical applications. As the 
main contribution of this work, an efficient algorithm for ex- 
haustively finding minimal stopping sets will be presented. By 
taking advantage of the sparse structure of the corresponding 
decoding tree, we are able to exhaust all stopping sets of size 
< 13 for LDPC codes of n = 512 and thus identify the 
minimal stopping distances that are < 13. For comparison, 
( ) w 2.5 x 10 25 trials are required if a brute force approach 
is employed, which demonstrates the efficiency of our scheme 
especially when there is no existing exhaustion algorithm for 
stopping sets of non-trivial sizes other than the brute-force 
search. Our algorithm will be generalized for trapping sets, and 
we will see that all trapping sets of size < 11 can be exhausted 
for n — 512 codes. The trapping set exhaustion algorithm also 
explains why good codes for BECs are generally good for 
other channels as well. 

In addition to its apparent computation-theoretic interest, 
the exhaustive search algorithm efficiently computes the exact 
value or a lower bound of the minimal stopping / trapping 
distance of arbitrary LDPC codes of practical sizes, which 
has the following three major applications: 

1) For binary erasure channels, an exhaustive list of min- 
imal stopping sets can be used to obtain a BER / FER 
lower bound that is tight in both order and multiplicity. 
For non-erasure channels, the list of small trapping sets 
can be used to derive lower bounds tightly predicting 
the error floor region [6]. 

2) One byproduct of our stopping set exhaustion algorithm 
is a BER/FER upper bound for any fixed arbitrary codes 
over BECs. With a pivoting rule matching the tree- 
based narrowing enumeration, an asymptotically tight 
upper bound can be obtained, which provides a worst 
performance guarantee for the regime with arbitrarily 



low erasure probability. 
3) Using the exhausting algorithm as an objective function 
in finite code optimization, good codes can be obtained 
and the performance is guaranteed without any com- 
binatorial outlier, a weakness of randomized construc- 
tions first pointed out in [6]. Further implementation 
of the code optimization algorithm and the suppressing 
effects of code lifting will be deferred to a companion 
paper [18]. 

Recent research related to the determination of the minimal 
Hamming distance, the pseudo-codeword weight, and the girth 
of the Tanner graph can be found in [38], [39], [40], [41], 
[42], [43]. Ensemble-based FER bounds for MAP decoders 
are explored in [44], [45], and the references therein. It is 
worth noting that although being highly correlated with the 
asymptotic performance [34], deciding the girth of the Tanner 
graph is not an NP-hard problem and can be easily determined 
with complexity 0(n 2 ). 

The remainder of this paper is organized as follows. This 
paper contains two major applications, stopping set exhaustion 
and trapping set exhaustion. Basic definitions of these types 
of error-prone patterns are given in Section II, and the NP- 
completeness of exhausting small trapping sets is proved in 
Section III. We then start from deriving a tree-based algorithm 
for computing the BER upper bounds for BECs in Section IV, 
which is then proved to be equivalent to a stopping set 
exhaustion algorithm based on narrowing search over the 
corresponding tree structure. Inspired by different properties 
of stopping sets and by the corresponding Boolean expression 
framework, several methods to further improve the efficiency 
of the proposed algorithm are discussed in Section V, which 
are the key elements of an efficient implementation. The 
same algorithm is generalized for trapping set exhaustion 
in Section VI. Section VII contains numerical experiments, 
including results for the (23,12,7) Golay code, the (155,64,20) 
Tanner code [29], the Ramanujan-Margulis (q = 13,p = 5) 
code [46], the Margulis code (p = 7) [47], and the two rate 
1/2, n w 500 lifted LDPC codes from the companion paper 
[18]. Section VIII concludes this paper. 

II. Preliminary & Formulation 
A. Code Representation 

For fixed n e N, let x = (ari, . . . , x n ) S {0, 1}™ denote 
a transmitted vector, for which n is the codeword length. 
The receiving signal vector denoted by y = (y\, . . . ,y n ) is 
generally the image of the transmitted x corrupted by noise; 
P(y|x) the conditional distribution of y given x describes 
the underlying channel. Throughout this paper, we consider 
only memoryless channels, i.e., P(y|x) = J\7=i P(Vi\ x i)- 
One particular type of memoryless channel is the binary 
erasure channel for which the received signal y is either 
the uncorrupted transmitted signal x or an erasure e, and a 
memoryless BEC can be completely described by its erasure 
probability e = P(y = e\x). 

A linear code is a collection of allowed transmission vectors 
C = {x} C {0, 1}™ that forms a linear subspace with addition 
and multiplication operations defined over the Galois field 
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Fig. 1. A simple parity check code Ci. 

GF(2)™. Any linear code C can be represented by an m x n 
parity-check matrix H which contains the basis vectors of the 
null space of C, and m is the dimension of the null space. 
I.e., if we let x be a column vector, then Hx = 0,Vx G C. 
It is worth emphasizing that this representation based on H 
is not unique. With all our results being H-dependent, we 
sometimes use the code C to refer to the corresponding parity- 
check matrix H instead of the collection of valid codewords 
{x}. 

An equivalent graphical representation of the code C can 
be obtained by regarding the m x n matrix H as the in- 
cidence matrix of a bipartite graph containing two sets of 
vertices: the variable nodes {vi, . . . ,v n } and the check nodes 
{a,..., Cm}, such that H yi = 1 if and only if there is an 
undirected edge connecting (cj,Vi). For example, a simple 
parity-check code C\ with 

/l 1 1 0\ 

10 10 1 

1110 

\1 1 1/ 

can be mapped bijectively to a bipartite graph as in Fig. 1, 
which is termed the Tanner graph of the corresponding parity- 
check code. Since the coded bits \x\, . . . ,x n } can be mapped 
to the variable nodes {vi, . . . ,v n } bijectively, they are gen- 
erally used interchangeably. However, in the most stringent 
definitions, {x\, . . . ,x n } refers to the "bits" of the code vector 
x satisfying Hx = while {v\, . . . ,v n } refers to the "nodes" 
in the corresponding Tanner graph. 

LDPC check codes are a special family of parity check 
codes introduced in the early 1960s [2], [3], for which the 
number of ones in H is limited to be of the order of n. When 
combined with the message-exchanging belief propagation al- 
gorithm [48], capacity-approaching error-correction capability 
has been reported. Although our results are derived with the 
application of LDPC codes in mind, the same results hold for 
general parity-check codes as well. 

B. Codeword Supports, Stopping Sets & Trapping Sets 

1 ) Codeword Supports: A codeword support is a subset of 
{xi, . . . , x n } such that if we set the values of all Xi's in the 
codeword support to 1 and all other a^'s to 0, the resulting 
vector x is a valid codeword. The Hamming weight of a 
codeword can thus be defined as the number of ones in the 
codeword or as the number of elements in the corresponding 
codeword support. The minimal Hamming distance g?h of 
a parity-check code C is defined as the minimal Hamming 
weight of all non-empty codeword supports. Since Hx = 
for all valid codewords, one can prove that a subset of 
{xi, . . . ,x n } is a codeword support if and only if the induced 



subgraph contains no check nodes with odd degrees. For 
example, {x\, x 2 , x e } is a codeword support of the code C\ 
depicted in Fig. 1. 

2) Stopping Sets: A stopping set is a subset of {vi, ...,«„} 
such that the induced subgraph contains no check nodes of 
degree one. Using code C\ in Fig. 1 as an example, both 
{v 2 , ^3,^4} and {vi, v 2 , V4,v 5 } are stopping sets for code C\. 
The minimal stopping distance d$ is defined as the minimal 
size of all non-empty stopping sets. By definition, a codeword 
support is always a stopping set but not vice versa, which 
implies that d$ < dn for all parity-check codes. By adding 
redundant parity check rows to the parity-check matrix H, 
one can improve the code structure so that all stopping sets 
are valid codeword supports and the minimal stopping distance 
equals the minimal Hamming distance [22]. 

Di et al. in [4] showed that for BECs, a belief propagation 
iterative decoder [48] fails if and only if the set of erased bits 
contains a stopping set. As a result, for small erasure proba- 
bility e, the FER performance is determined by the minimal 
stopping distance d$ and the corresponding multiplicity mg, 
the latter of which is defined as the number of distinct stoping 
sets of size dg. 

3) Trapping Sets: The notion of trapping sets stems from 
an operational definition in which a trapping set originally 
refers to a subset of {vi, ...,?)„} that is susceptible to errors 
under message-exchanging iterative detectors in the following 
way. If the observations on these nodes are misleading, it is 
"highly unlikely" for the extrinsic messages to correct any 
of the nodes. The errors are "trapped" within these nodes 
[6]. It is a concept depending both on the underlying channel 
model and the decoding algorithm. For example, a stopping 
set is legitimately also a trapping set when the underlying 
channel model is the BEC and the belief propagation decoder 
is employed. Empirically, almost all trapping sets are near- 
codewords when channels other than BECs are considered 
[7]. To be more explicit, almost all trapping set are a subset 
of {vi, ...,?)„} such that the induced subgraph contains only 
a "limited" number of check nodes of odd degree and are 
thus "nearly" a codeword support. A near-codeword can be 
categorized by two numbers, the total number of variable 
nodes involved and the number of check nodes of odd degree 
in the induced graph. For example, a (12,4) near-codeword 
refers to a set of 12 variable nodes for which there are 4 
check nodes of odd degree in the induced subgraph. To better 
facilitate our discussion, we redefine the trapping set from 
the graph-theoretic perspective, which is independent of the 
underlying channel model and the decoding algorithm: 

Definition 1 (k-out Trapping Sets): A subset of 
{vi ,...,«„} is a fc-out trapping set if in the induced 
subgraph, there are exactly fc check nodes of degree one. 

By the above definition, every stopping set is a 0-out 
trapping set. {vi, v 2 , V4} in C\ of Fig. 1 is a 1 -out trapping 
set. It is worth noting that the definition of a fc-out trapping 
set is slightly different from the definition of near-codewords. 
The near-codeword focuses on the total number of check nodes 
of odd degree, including those with degree 1 and those with 
degrees > 3, and emphasizes its link to valid codewords. The 
fc-out trapping set considers a pattern resembling the stopping 
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sets but has fc edges connecting to check nodes of degree 1 
that are able to receive correct extrinsic messages to recover 
the contaminated bit values. In addition to the k check nodes 
of degree 1, there might be more check nodes of odd degrees 
> 3 within a fc-out trapping set, which makes the definition 
different than a (s, k) near codeword. This slight relaxation 
does not affect the generality of our exhaustion algorithm for 
the following two reasons. For any set of s variable nodes, 
if it is an (s,t) near-codeword, then, by definition, it must 
be a fc-out trapping set for some fc < t. As a result, any 
attempt to exhaust all fc-out trapping sets of sizes s < s and 
fc < to can also serve the purpose of exhausting all (s,t) 
near codewords for s < so and t < to- Furthermore, for 
a randomly constructed code of large length n, almost all 
small stopping sets are codewords and almost all small fc- 
out trapping sets of size s are (s, fc) near-codewords due to 
some combinatorial bias [18]. In our numerical examples, all 
minimal fc-out trapping sets are (s, fc) near-codewords. 

One can similarly define the minimal fc-out trapping dis- 
tance dx,fc as the minimal size of all non-empty fc-out trapping 
sets. 

III. NP-COMPLETENESS 

A. Existing Results 

One important class of problems in the theory of compu- 
tation is the P-class problems such that all problems of this 
type can be solved by a modern computer within (P)olynomial 
time Poly(ra) steps with respect to the input size n, assuming 
sufficient amount of memory is available. On the other hand, 
the class NP problem refers to those problems that are guar- 
anteed to be solvable by a fictional (N)on-deterministic Turing 
machine within (P)olynomial time steps. Or equivalently, the 
class NP problems refer to those problems with solutions that 
can be "verified" by a modern computer within polynomial 
time steps. A class of problems is NP-hard if all other NP 
problems of size n can be "reduced" within Poly(n) time so 
that the algorithm of the original problem can be used as 
a solver for these other NP problems. There is no known 
efficient Poly(n) time algorithm for solving any NP-hard 
problems while the existence / nonexistence of such algorithms 
has been one of the most important open problems within 
the mathematics and computer science community [49] for 
more than three decades. A problem is considered to be 
NP-complete if it is both NP and NP-hard. More rigorous 
definitions / discussions of these concepts can be found in 
[50]. 

Some existing NP-complete problems in the coding and 
information theory literature are as follows. 

1) Minimum Hamming Distance: A binary decision prob- 
lem is a special type of problem in which a yes/no 
question is proposed and the algorithm/solver outputs 
1 if the answer to the proposed question is positive and 
outputs if the answer is negative. We consider the 
following decision problem: given as input an arbitrary 
parity-check matrix H and an integer t, the decision 
problem MIN-DISTANCE(H, t) proposes the question 



of whether (in < t, in which g?h is the minimal Ham- 
ming distance of the parity-check code C specified by 
H. It has been shown in [24], [25], [26] that the decision 
problem MIN-DISTANCE(H, t) is NP-complete and the 
question of whether dn < t is hard to answer for H of 
large dimension. 
Note: One can easily show that this decision problem MIN- 
DISTANCE(H, t) is of similar complexity to the problem of 
directly computing du given H, 1 while the simpler nature of 
the Yes/No question facilitates the complexity analysis. 
2) Minimum Stopping Distance: Given as input an ar- 
bitrary parity-check matrix H and an integer t, the 
decision problem STOPPING-DISTANCE (H, t) pro- 
poses the question of whether ds < t, in which d$ 
is the minimal stopping distance of the parity-check 
code C specified by H. For notational simplicity, we 
use SD(H, t) as shorthand of the decision problem 
STOPPING-DISTANCE(H, t). It has been shown in 
[23] that SD(H,i) is also an NP-complete problem. 

One may limit the input parity-check matrix H to be 
a sparse matrix H sparsc such that the number of ones in 
H sparso is of the order 0(n). The new decision problem 
SD(H sparso , t) may have different complexity than the 
original SD(H, t) due to this additional constraint on H. 
[23] shows that SD(H sparsc , t) is still an NP-complete 
problem regardless of such a sparsity constraint. 

B. Minimum k-out Trapping Distance 

Given as input an arbitrary parity-check matrix H and 
an integer t, the decision problem fc-OUT-TRAPPING- 
DISTANCE(H, t) proposes the question of whether c?x,fc < t, 
in which dx.fc is the minimal fc-out trapping distance of the 
parity-check code C specified by H. For notational simplicity, 
we use fc-OTD(H,i) as shorthand of fc-OUT-TRAPPING- 
DISTANCE(H, t). We can then prove the following theorem. 

Theorem 1: For any fixed integer fc, the decision problem 
fc-OTD(H, t) is NP-complete. With a sparsity constraint on the 
input matrix H sparsc , the decision problem fc-OTD(H sparsc , t) 
is still NP-complete. 

Before proving Theorem 1, we first generalize the existing 
NP-completeness results on deciding the minimal stopping 
distance. Consider a new decision problem SD(H, t,Vi) with 
an additional input Uj. Similar to SD(H, t), SD(H,t, v{) de- 
termines whether there is a non-empty stopping set containing 
Vi and of size < t. SD(H, t, Vi) can be thus regarded as 
deciding the bit-wise minimal stopping distance of bit Xi. 
We can further generalize the above concept and consider 
another decision problem SD(H, t, {vi x , . . . , Vi k }) with the 
additional input being any subset of {v\, . . . ,v n }. Analo- 
gously, SD(H, t, {vi 1 , . . . , Vi k }) decides whether there is any 
stopping set containing {v il , . . . , Vi k } as a subset and of size 
< t. We then have the following lemma. 

Lemma 1: All of SD(H,i), SD(H,i,^), and 
SD(H, t, {vi 1 , . . . , Vi k }) are NP-complete problems. 

'There are instances in which the decision problem is strictly easier 
than direct computation, for example, the primality test problem vs. the 
factorization problem. 
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Algorithm 1 Poly-time reduction of £>OTD(H,i) to 
SD(H,t, {v n ,...,v lk }) 

1: Input: H and t. 

2: <- 0. 

3: repeat 

4: Based on the Tanner graph, select fc edges connect- 
ing to fc distinct check nodes and denote them as 

{ > Cji ),..., (l>u,CjJ}. 

5: if there is no edge between Vi , and Cj yl for any k' ^ k", 
then 

6: Construct a new H' by removing those columns i in H 
simultaneously satisfying (1) i / i a ,Va = l,...,k, and 
(2) 3o e {1, . . . , fc} such that H jai = 1. 

7: Construct a new H" by removing row j a ,Va = l,...,k 
from H'. 

8: <P^SV(H",t,{v n ,...,v lk }). 
9: end if 

10: until cf> — 1 or all possible selections of k distinct edges are 

exhausted. 
11: Output: 4> 



Algorithm 2 Poly-time reduction of SD(H,i) to LESS-fc- 
OTD(H, t) 



1: Input: H and t. 

2: Denote the corresponding bipartite Tanner graph by 

({vi, ■ • ■ , v„}, {ci, . . . , Cm}, E) where E contains the 

edges of the graph. 
3: Construct a H' by duplicating (fc + 1) copies of the original 

Tanner graph, and denote the (fc + l)n variable nodes as {v t y : 

Vi = l,...,n,Vfc' = l,...,(fe + l)}. 
4: for i = 1 to n do 

5: Consider the (fc+1) duplicated variable nodes v i>k i . Add fc(fc+ 
l)/2 new check nodes of degree 2 and fc(fc + 1) new edges 
so that the shortest distance between any Vi,k ± and Vi,k 2 is 2, 
and the shortest path between them is unique and uses exactly 
one degree 2 check node. Namely, the (fc + 1) variable nodes 
now form a "clique" with the duplicated nodes linked to each 
other by a degree 2 check nodes. 

6: end for 

7: Denote the final graph as H". 

8: Output: LESS-fc-OTD(H",t(fc + 1)) 



With a sparsity constraint on the input matrix 
H sparsc , all of SD(H sparsc ,i), SD(H sparso ,i,Wi), and 
SD(H sparso , t, {v h ,...,v ik }) are still NP-complete. 
The above result shows that it is of similar complexity to 
decide the minimal stopping distance regardless of whether it 
is from a frame-wise perspective, from a bit-wise perspective, 
or from a subset-wise perspective. 

Proof: We prove this lemma by showing that these three 
problems can be reduced to each other. The reduction from 
SD(H, t) to SD(H, t, Vi) is straightforward and one needs only 
to perform SD(H, t, Vi) n times for all i = 1, . . . , n. Output 1 
if 3i such that SD(H, t, returns 1, and output otherwise. 
The converse reduction from SD(H, t, Vi) to SD(H, t) is as 
follows. Without loss of generality, we assume the input Vi 
being v n , the last variable node. Construct a (m + n — 1) x n 
matrix H' from H by appending (n — 1) rows of zeros at the 
bottom of H. In the extra (n— 1) rows, set H'- n = H'- -_ m = 1 
for j = m + l,...,m + n—l. With this construction, one can 
easily show that any stopping set with respect to the matrix H' 
must be a stopping set of H containing v n and vice versa. The 
reduction from SD(H,£, Vi) to SD(H',t) is thus obtained. 

The reduction from SD(H, t, Vi) to SD(H, t, {vi 1 , . . . , Vi k }) 
can be achieved by letting the subset be {vi}. The converse 
reduction from SD(H, t, {v il , . . . , v ik }) to SD(H, i, Vi) is as 
follows. Construct a (m + k) x (ra + 1) matrix H' by appending 
an all-zero column and k all-zero rows at the right and the 
bottom of H respectively. Set H'- n+l = H'-^. ^ = 1 for j = 
m + 1, . . . , m + k. With this construction, v s C {v\, . . . , v n } 
is a stopping set of H with {vi 1 , . . . , Vi k } C v s if and only 
if v s U {fn+i} is a stopping set of H' containing v n +\. The 
reduction from SD(H, t, {v il , . . . , v ik }) to SD(H', t+1, w„+i) 
is thus obtained. With the above four reductions, the proof of 
Lemma 1 is complete. 

■ 

With the help of Lemma 1, the proof of Theorem 1 can be 
described as follows. 

Proof of Theorem 1: Consider an auxiliary decision 
problem LESS-fc-OTD(H, t) that decides whether there is any 



minimal fco-out trapping distance rfT.fe with ^T,fc < t and 
fc < k. We first show that (i) LESS-A:-OTD(H, t) can be 
reduced to fc-OTD(H, t) in polynomial time, (ii) fc-OTD(H, t) 
can be reduced to SD(H, t, {vi 1 , . . . , Vi k }) in polynomial time, 
and (iii) SD(H,£) can be reduced to LESS-fc-OTD(H, t) in 
polynomial time. 

> The first reduction (i): For any fixed k, the answer 
to LESS-fc-OTD(H, t) can be obtained by solving k'- 
OTD(H, t) for all k' < k. Output 1 if any fc'-OTD(H, t), 
k' < k, returns positive, and output otherwise. The 
correctness of this reduction is self-explanatory. 

• The second reduction (ii) is described in Algorithm 1. It 
is straightforward to show that this reduction takes only 
polynomial time. Its correctness can be proved by noting 
that any stopping set v s of H" containing {vi 1 , . . . , Vi k } 
must be a fc-out trapping set of H, and vice versa. 

> The third reduction (iii) is described in Algorithm 2. The 
polynomial time complexity of this reduction is straight- 
forward. The correctness of this reduction is proved as 
follows. 

Suppose there exists a stopping set v s of H with size 
< t. Let v S) i, . . . , v s ^ + i denote the duplicated (k + 1) 
copies of v s in H". Then (J»=i v s.i i s a stopping set of 
H" with size < t(k + 1). LESS-/c-OTD(H", t(k + 1)) 
must return 1 . 

Conversely, suppose LESS-fc-OTD(H", t(k + 1)) re- 
turns 1. There exists a k -out trapping set v" of H" 
with size < t(k + 1) and fc < k. Without loss of 
generality, we may assume v\ t \ G v". We then focus 
on the clique formed by {v\^> '■ k' = 1, . . . , k + 1} and 
the corresponding k(k + l)/2 degree 2 check nodes. We 
would like to show that all v\ t k' must be in v". Suppose 
there exists a k' such that v\^' £ v". By noting that the 
maximal flow from vi t i to v\ t k' is k and by the max- 
flow min-cut theorem, all cuts must include at least k 
edges. Therefore there exist > k check nodes of degree 
2 with one neighbor in v" and the other not in v". The 
sub-graph induced by v" must have > k check nodes of 



6 



degree 1, which leads to a contradiction. Therefore, all 
wi fe/ are in v", and v" must be a union of all duplicated 
copies of variable nodes v S; j, namely, v" = Uj=i v s,v 
If v Sji is not a stopping set, then there exists at least 
one degree 1 check node in the subgraph of H induced 
by v s ^. Therefore, there must be at least k + 1 degree 1 
check nodes in the subgraph of H" induced by v", which 
contradicts the assumption that v" is a fc -° ut trapping 
set with k < k. From the above reasoning, v s i must be 
a stopping set of H. The original problem SD(H,t) is 
reduced to LESS-fc-OTD(H", t(k + 1)). 
By invoking the NP-completeness results in Lemma 1, 
Theorem 1 is proved. 2 



IV. Exhausting the Stopping Sets 

In this section, we focus mainly on exhausting the stopping 
sets for the BEC while the generalization to exhausting fc-out 
trapping sets will be deferred to Section VI-D. 

The fact that the iterative decoder fails when the erased 
bits contain a stopping set implies that each stopping set 
v s corresponds to a lower bound el Vs l on the FER where e 
is the erasure probability. Exhausting stopping sets of sizes 
smaller than or equal to K is thus equivalent to eliminating 
the possible existence of lower bounds of order 0(el K l). 
On the other hand, an alternative way of eliminating possible 
FER lower bounds of order O is to construct an upper 

bound O (e\ K+1 \) if such an upper bound exists. Sharing the 
same effects as eliminating possible lower bounds, it can be 
thus shown that any FER upper bound is able to serve as an 
exhaustion algorithm for small stopping sets and vice versa. 
For the following, we construct an efficient algorithm for 
computing upper bounds on the error probability for arbitrary 
parity-check codes over BECs, which is later shown to be 
equivalent to an efficient stopping set exhaustion algorithm in 
Section V-C. 

A. The Boolean Expression Framework 

Without loss of generality, we assume the all-zero codeword 
x = is transmitted for notational simplicity. With this 
assumption and with the BEC being considered, a decoding 
algorithm for bit xu i £ {1, . . . , n} is a function outputting 
either or e, the former of which represents Xi being suc- 
cessfully recovered / decoded and the latter of which means 
the bit-wise decoder fails. To be more explicit, a decoding 
algorithm for bit Xi is a function : {0, e}" i-> {0, e} 
with output Xi = fi(y), where y £ {0, e}" is the received 
signal vector. In this paper, /, ; (, Vi £ {1, . . . , n}, is used to 
denote the iterative decoder for bit Xi after I iterations. And 
we use fi = lim^oc to denote the end decoding result 
after an infinite number of iterations, or equivalently, the end 
result after the iterative decoder stops improving. If we further 
relabel the element "e" by "1," /, and become Boolean 

2 The language of proving reductions herein is a simplified version of the 
language commonly used in the theory of computation. All our statements 
can be made rigorous using the formal language of reduction. 



functions, and the BER for bit Xi is simply the expectation 
Pi = E{/j(y)}. Another advantage of this conversion is that 
the iterative message map at the variable node then becomes 
the binary AND operation, and the iterative message 
map at the parity check node becomes "+", the binary OR 
operation. This Boolean-expression framework was implicitly 
considered in [51], [10] and many other papers discussing 
iterative decoders on BECs, but the power of it has never been 
fully utilized. Later in this section, many useful implications 
will be derived under this framework. 

Take the simple parity-check code C\ in Fig. 1 as an 
example. Suppose we further use fi^j.i to represent the 
message from variable node Vi to check node cj during the 
l-th iteration. The iterative decoders f 2 j, I £ {0, 1, 2}, for bit 
x 2 then become 

/2,o(y) = 2/2 

/2,i(y) = 2/2(2/1 + 2/3) (2/4 + ye) 

/2,2(y) = 2/2 (2/l(./5^4,l + /6^4,l) + 2/31/4^3,1 + /5^3,l)) 

• (2/4(/3^3,l + /5^3,l) + 2/6(jW,l + /5^4,l)) 

= 2/2 (2/1(2/5 + 2/6 ) + 2/3(2/4 + 2/5)) 

• (2/4(2/3 + 2/5) + 2/6(2/1 + 2/5)) • (1) 

As the iteration proceeds, we can derive the Boolean expres- 
sion for all f 2 j(y), I £ N. The final decoder of bit x 2 is 
/2 := limj^oc f2.i, and in this example of C\, one can verify 
that / 2 (y) = / 2 , 2 (y) for all y £ {0, 1}". Although (1) admits 
a beautiful nested structure, the repeated appearance of many 
Boolean input j/j's, also known as short "cycles," poses a great 
challenge to the evaluation of the BER p 2 = E{/ 2 (y)}. One 
solution is to simplify (1) by expanding the nested structure 
into a sum-product Boolean expression [10]: 

/2(y) = /2,2(y) = 2/12/22/6 + 2/22/32/4 + 2/12/22/42/5 + 2/22/32/52/6- 

(2) 

E{/ 2 (y)} can then be evaluated by the projection algebra in 
[10] or equivalently by the inclusion-exclusion principle. For 
this example, we have p 2 (e) = 2e 3 + 2e 4 — 5e 5 + 2e 6 , where 
e = E {yi},\/i, is the erasure probability. For comparison, with 
e = 0.1, p 2 (e) = 2.152 x 10~ 3 while the BER predicted by 
density evolution after 2 iterations is w 1.4 x 10~ 4 . An order- 
of-magnitude gap is observed between the actual performance 
and the prediction by density evolution due to the presence of 
many short cycles. 

Define the "irreducible stopping set" as a stopping set 
containing no other non-empty stopping set as its proper 
subset, which differs from the definition of the minimal 
stopping sets (the stopping sets of minimal size). All minimal 
stopping sets are irreducible stopping sets but not vice versa. 
It can be proved that in the simplified sum-product form, 
each product term can be mapped bijectively to an irreducible 
stopping set. For example, the product term 2/i2/22/42/5 m (2) 
corresponds to an irreducible stopping set {v\, v 2 , V4, v^}. The 
product term y^y^ ■ ■ ■ Ui k and the irreducible stopping set 
{v il , Vi 2 , . . . , v ik } will thus be used interchangeably. 

Instead of constructing the exact expression of /j(y), in 
which the number of product terms is exponential with respect 
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to n, if only a small collection of the product terms is 
identified, say 2/12/22/6 an d 2/22/32/4, then a lower bound 

E{/ LB , 2 (y)} = 2e 3 - e 5 < E{/ 2 (y)} = p 2 (e), Ve e [0, 1] 

can be obtained where 

/lb,2(v) = 2/12/22/6 + 2/22/32/4 < /2(y), Vy G {0, 1}". 

In our example, E{/LB.2(y)} = 2e 3 — e 5 is asymptotically 
tight and determines the error floor. If any one of the minimal 
stopping sets, which are 2/12/22/6 and 2/22/32/4 m this example, 
is not identified, then the resulting lower bound will be loose. 
The major challenge of this approach is thus to ensure that 
all minimal stopping sets are exhausted. Furthermore, even 
when all minimal stopping sets are exhausted, this lower bound 
is tight only in the high signal-to-noise ratio (SNR) regime. 
Whether the SNR of interest is high enough can be determined 
only by Monte-Carlo simulations and by extrapolating the 
waterfall region. An upper bound thus becomes essential for 
a deeper understanding of the code behavior. 

An upper bound can be constructed by iteratively computing 
the sum-product form of /2,z+i(y) from that of /2,z(y)- To 
counteract the exponential 3 growth rate of the number of 
product terms with respect to I, during each iteration, we can 
"relax" and "merge" some of the product terms so that the 
complexity is kept manageable [10]. For example, /2,2(y) in 
(2) can be relaxed and merged as follows. 

/2,2(y) = 2/12/22/6 + 2/22/32/52/6 + 2/22/32/4 + 2/12/22/42/5 

< 12/22/6 + 2/ 2 l2/6 + 2/2I2/4 + I2/22/4I 

= 2/22/6 + 2/22/4 
A 

— /UB,2, 

so that the number of product terms is reduced from four to 
two. This technique is generally very loose since the relaxation 
step destroys much information. To generate tight results, the 
number of relaxation steps must be kept minimal, which yields 
again an exponential growth rate of the number of product 
terms. As a result, good/tight results have been reported only 
for codes of lengths n < 20. 

In contrast, we construct an efficient upper bound UB^ > 
E{/j(y)} by preserving much of the nested structures and 
tight upper bounds can be obtained for n = 300-500, a 
significant improvement considering the NP-hardness of this 
problem. Furthermore, the tightness of our bound can be 
verified with ease, a feature absent in the relax-and-merge 
approach. Combined with the lower bound E{/LB,i(y)}, the 
finite code performance can be efficiently bracketed for the 
first time. 

B. The Iterative Decoding Functions 

Some useful notation is as follows. We use yf to represent 
a partial segment (yi,yi +1 , ...,y k ) from yi to y k . With this 
notation, y = yi _1 0y™ +1 represents a received vector with the 
i-th value yi being 0. For any i, we define yi as a determining 

3 The corresponding exponential order is generally 
O (((<it> — l)(d c — l)) 1 ), where d v and d c are the maximum variable and 
check node degrees. 



variable of /(y) if 3y\-\y? +1 such that /(y^Oy^) 

/(yr'iyf+i). 

Definition 2 (Iterative Decoding Functions): A Boolean 
function /(y) is an iterative decoding function if /(y) can 
be represented as a series of combinations of yi, . . . , y n , the 
binary AND "•" and the binary OR "+" operations without 
using the binary NOT "-1" operation. 

For example: j/i + 2/2(2/1 + 2/3) is an iterative decoding 
function while the XOR function 2/i( _, 2/2) + ( _, 2/i)2/2 is not. All 
/j(y)> fi,l{y)> an d /nj,i(y) discussed in the previous section 
are iterative decoding functions. Before introducing a simple 
upper bound, we first prove the following two properties of 
iterative decoding functions. 

Proposition 1 (Monotonicity): Suppose /(y) is an iterative 
decoding function. Then /(y) is monotonically increasing 
with respect to each input variable yi. That is, 

Vi,Vyj-\Vy? +1 , /(yr'Oyr+i) < /(yl^y^). 

Proof: Since each variable node corresponds to an 
AND and each check node corresponds to an OR, it can be 
shown that any iterative decoding function can be converted 
to a parity-check code detection problem assuming iterative 
decoders. With one more uncorrupted observation y t , the 
iterative decoder can only perform better. Proposition 1 is 
simply a restatement of the channel degradation argument. 
This property, however, differentiates the iterative decoding 
function /(y) from arbitrary Boolean functions. ■ 

Proposition 2 (Positive Correlation): The correlation be- 
tween two iterative decoding functions /(y) and </(y) is 
always non-negative, i.e., E{/(y)c/(y)} > E{/(y)}E{c/(y)}. 

Proof: We prove this result by induction on the num- 
ber of common determining variables of /(y) and </(y). 
When /(y) and c?(y) share no common determining variable, 
then /(y) and c?(y) are independent and E{/(y)c?(y)} = 
E{/(y)}E{ 5 (y)}. 

Suppose the same inequality holds for /(y) and g(y) 
sharing k determining variables. Consider the case in which 
/(y) and g(y) share k + 1 determining variables. Without 
loss of generality, let 1/1 be one of the common determining 
variables. Given y\ — (or y\ = 1), the conditional /(y) and 
g(y) share only k common determining variables. We then 
have 

E{/(y).g(y)} 

= P(yi = l)E{/(y) fl (y)|yi = l} 

+ P(2/i=0)E{/(y) 5 (y)|2/i =0} 
> P(yi = l)E{/(y)|2/i = 1}E {g(y)\ Vl = 1} 

+ P(yi=0)E{/(y)|y 1 =0}E{ fl (y)|y 1 = 0} ) (3) 

in which the last inequality follows from induction. On the 
other hand, 

E{/(y)}E{ 5 (y)} 

= (P (2/1 - 1)E {/(y)|2/i = 1} + P (27i = 0)E {/(y)| yi - 0}) 
• (P (2/1 - 1)E {.g(y)|2/i = 1} + P (2/1 = 0)E{g(y)\ yi - 0}) . 

(4) 
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Subtracting (4) from (3), we have 

(3)-(4)>P(yi = l)P(i/i=0) 

■ (E{/(y)|i/i = 1} - E {/(y)|yi 
■(E{5(y)|yi = i}-E{ 5 (y)|y i: 



= 0}) 
0}). 



By Proposition 1, the right-hand side of the above inequality 
is always non-negative. It is thus proved that E {/ (y)g(y)} > 
E {/(y)} E {ff(y)} for /(y) and g(y) sharing k + 1 common 
determining variables. By induction, Proposition 2 is proved. 



C. A Simple Upper Bound Based on Tree-Trimming 

Let g(y) and h(y) be two iterative decoding functions. 
Consider the simple variable node message map /„ = g ■ h and 
the simple check node message map f c = g + h respectively. 
The three simplest cases are discussed as below and three rules 
are introduced to cope with these scenarios. 

Rule — Density-Evolution-Like Evaluation 

If g{y) and h{y) share no determining variable, then g(y) 
and h(y) are independent and 

E{/„(y)} = E{g(y)h(y)} = E{g(y)}E{h(y)} 

E{/ c (y)} = E{.g(y) + My)} 

= E{ 5 (y)} + E{/ l (y)}-E{ 5 (y)}E{/ i (y)}. 

Namely, when the inputs are independent the BER of interest 
can be iteratively computed by the above formulas. Nonethe- 
less, a more interesting question is for the cases in which 
we have dependent g(y) and h(y). Can we still use this 
density-evolution-like (DE-like) evaluation as an upper bound? 
The answer to this question is addressed in Rules 1 and 2 
respectively. 

Rule 1 — A Simple Relaxation 

Suppose g(y) and h(y) share at least one determining 
variable, i.e., there are repeated nodes in the input arguments 
of g(y) and h(y). By the inclusion-exclusion principle and by 
Proposition 2, we have 

E{/ c (y)} = E{ 5 (y)+My)} 

= E{g(y)} + E{h(y)}-E{g(y)h(y)} 

< E{g(y)} + E{h(y)} E{g(y)}E{h(y)}. (5) 

The above rule suggests that when the incoming messages 
<?(y) and h(y) of a check node are dependent, the error 
probability of the outgoing message can be upper bounded by 
blindly assuming the incoming messages are independent and 
by invoking Rule 0, the DE-like evaluation. Furthermore, the 
upper bound computed by Rule 1 has the same asymptotic 
order as the target error probability, as can be easily seen 
in (5). The multiplicity of the upper bound may be different 
from that of the error probability. Due to the random-like 
interconnection within the code graph, for most cases, g(y) 
and h(y) are "nearly independent" and the multiplicity loss is 
not significant. The realization of Rule 1 is illustrated in Fig. 2, 
in which we assume that yi is the only common determining 
variable. 




(a) Original (b) Relaxation 

Fig. 2. Rule 1: A simple relaxation for check nodes. 




(a) Original (b) Decoupled 

Fig. 3. Rule 2: A pivoting rule for variable nodes. 

Rule 2 — The Pivoting Rule 

Consider the simplest case in which g(y) and h(y) share 
one and only one common determining variable y\ . By simple 
Boolean algebra, we can upper bound f v (y) = g{y) • h(y) as 
follows. 

g(y) • My) < ff(OyJ) • MOyJ) + yifl(iyj) ■ h(iy%). 

By Proposition 1, the inequality becomes equality and we have 
fv(y) = <?(0y£) • h(0y%) + Vl ■ ff (lyJ) • fc(lyj). (6) 

The realization of the above equation is demonstrated in 
Fig. 3. Once the tree in Fig. 3(a) is transformed to Fig. 3(b), 
messages entering the two variable nodes in Fig. 3(b) become 
independent since for fixed values of y\, the conditional 
functions giyiy^) and /i(yiy2 ) share no common determining 
variable and are independent to each other. The expectation of 
the variable node AND operation can be exactly evaluated by 
Rule 0. Although the two messages entering the top check 
node in Fig. 3(b) are still dependent, the expectation at the 
check node OR operation can be upper boundedby reapplying 
Rule 1 . The final upper bound thus becomes 



E{/*(y)} < E{/„(0y£)} + E{i, 1 }E{/„(lyJ} 
-E{.A,(0 y £)}E{ yi }E{A(ly£)}, 



(7) 



where for all y\ £ {0, 1}, 

E{/„(i/iy£)} - E{g{ yi y n 2 )}E{h{ yi y n 2 )}. 

In sum, after performing the pivoting rule as described by (6) 
and in Fig. 3, the expectation E{/„(y)} can also be upper 
bounded by the DE-like evaluation. 

Remark 1: The pivoting rule (6) is an equality and does 
not incur any performance loss. Any potential looseness of 
this upper bound (7) results from the application of Rule 1. 
Rule 2 thus preserves the asymptotic order of E{/„(y)} as 
does Rule 1, and only the multiplicity term might be loose. 

Remark 2: Although Rules to 2 hold for iterative decoding 
functions in sum-product forms as well, it is more efficient 
to represent an iterative decoding function in its original tree 



9 




(a) Original (b) After pivoting 

Fig. 4. The pivoting rule, Rule 2, in a real decoding tree. For the most 
general case, yca(v k ) can be an active variable node v with free input y 
or an inactive node with input hardwired to or 1, which is represented by 
the horizontal arrow entering yca(v k ). yca(vk) can also have > 2 children 
and the children not involved during pivoting are represented by lines entering 
yca(ffc) with different slopes. The function /(y) focuses mainly on the AND 
operation on the two incoming messages g(y) and h(y) with respect to the 
directions of v k and v' k . 

form since the latter admits efficient computation of the DE- 
like evaluation. For general iterative decoding functions, the 
DE-like evaluation can only be used as an "approximation" 
of the finite code performance without concrete justification. 
Rules 1 and 2 provide a mechanism for converting the original 
tree to a suitable format such that the direct DE-like evaluation 
gives us a rigorous upper bound on the finite code BER. 

D. The Algorithm 

As a simple (easy to evaluate) and powerful (tight in 
asymptotic order) upper bound, Rules to 2 are designed to 
upper bound the expectation of single operations, either an 
AND or an OR operation, with the incoming messages g(y) 
and h(y) sharing at most one common determining variable. 
These apparent limitations on the size of the tree and on 
the number of common determining variables make them not 
directly applicable to real decoding problems for parity-check 
codes. Nonetheless, once carefully concatenated, they can be 
used to construct UB^ for the infinite-sized decoding tree with 
many repeated nodes while still preserving most of the nested 
structure. We then have the following theorem: 

Theorem 2: Algorithm 3, a concatenation of Rules to 2, 
is guaranteed to find a BER upper bound UBi > pi for the 
infinite-sized decoding tree of bit Xi. 

Some technical details of this algorithm are discussed in 
Appendix I for interested readers. The proof of Theorem 2 
involves the graph theoretic properties of yca(xj ) and will be 
deferred to APPENDIX II in the interest of streamlining the 
discussion. 

This tree-based approach corresponds to a narrowing search 
of stopping sets. By denoting /r,t(y) as the corresponding it- 
erative decoding function based on the tree T after t iterations 
of the REPEAT-UNTIL loop in Line 3, we have the following 
theorem. 

Theorem 3 (A Narrowing Search): For any subset x C 
{x 1 , . . . , x„}, let y x = {yi,...,y n ) G {0, 1}™ be the cor- 
responding binary vector such that Mi G {1, . . . , n}, yi = 1 if 



Algorithm 3 A tree-based method for upper bounding p iy the 
BER of bit Xi. 

1: Notation: A variable node is active iff it has a free input observation y. 

2: Initialization: Let T be a tree containing one active variable node Vi 
with free input observation yi and all the neighboring check nodes of Vi, 
denoted as Cj . Vi is used as the root of T and all cj 's are the immediate 
children of v\. 

3: repeat 

4: Locate the next to-be-added variable node (also referred to as a leaf) 
according to the Tanner graph of the code. Suppose the next to-be-added 
leaf is a v k node with free observation y k . 
5: Add the new leaf v k to the tree T and set its status to be active. 
6: Construct the immediate check node children of v k based on the Tanner 

graph and add them to the tree T. 
7: if there exists at least another active variable node in T that has free 

input observation y k then 
8: Let yca(i)fe) denote the youngest common ancestor between the 
newly added leaf v k and any existing active node in T that has free 
input observation j/j. . We use v' k to denote the corresponding existing 
variable node in T such that Vk and v' k jointly result yca(ufc). 
9: if yca(ffc) is a variable node then 

10: As suggested by Rule 2, a pivoting construction involving tree 

duplication is initiated. Use yca(v k ) as the root for function /(y), 
the two incoming messages along the directions of v k and v' k are 
g(y) and h(y) respectively, and the shared determining variable 
is y k . The pivoting rule with respect to /(y), g(y), h(y), and y k 
is illustrated in Fig. 4(a-b). 

1 1 : Deactivate all currently active variable nodes in the duplicated left 

and right subtrees that have free observation y k by hardwiring their 
free observation y k either to 1 or to depending on whether the 
active variable node of interest is in the left subtree or in the right 
subtree. 

12: Use Fig. 4(b) as a reference. For the root yca(v k ), keep its original 

status. For the right duplicated subtree, deactivate its root, a newly 
constructed auxiliary variable node, by assigning a fixed input 
observation 1. For the left duplicated subtree, set the status of 
its root, also a newly constructed auxiliary variable node, to be 
active by assigning a free input observation y k . 

13: end if 

14: end if 

15: until the size of T exceeds the preset limit. 

16: Locate all to-be-added leaf variable nodes, add them to T, and deactivate 
them by hardwiring their corresponding free observation y to 1. 

17: UBi can be computed iteratively by the formulas in Rule 0. Namely, all 
incoming edges are blindly assumed to be independent. 



Xi € x and yi = otherwise. Let 

X t := {x C {xi, . . . ,x n } : /r,t(y x ) = 1}, 

and let X Xi denote the collection of all stopping sets contain- 
ing bit Xi. We then have 

X Xi cx t+1 cx ( ,VteN. 

In other words, at each time t, the tree T represents an 
exhaustive collection of candidates of stopping sets containing 
Xi and that collection narrows as iterations proceed. 
The proof of Theorem 3 is based on an incremental tree- 
revealing argument and is included in APPENDIX III. 

We conclude this section by listing some other properties 
of Algorithm 3 as follows. 

• The only computationally expensive step is when Rule 2 
is invoked, which, in the worst case, may double the tree 
size and thus reduces the efficiency of Algorithm 3. Some 
tree-pruning rules will be introduced later in Section V- 
A, which alleviates the computational cost of Rule 2 
significantly. 
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• Rule 1, being the only relaxation rule, saves much com- 
putational cost by ignoring repeated nodes. A lossless 
pivoting rule similar to Rule 2 can be devised based on 
the same principle. However, the tree structure will then 
be overly dissected into minute cases, which is equivalent 
to directly deriving the sum-product expression in (2) 
and is of prohibitive computational cost. The complexity- 
reduction of Rule 1 is a major component of the upper 
bounding algorithm. 

• Once the tree construction is completed, evaluating UB^ 
for any e G [0, 1] can be achieved efficiently with 
complexity O (|T| log(|T|)), where \T\ is the size of T. 

« The preset size limit of T provides a tradeoff between 
computational resources and the tightness of the resulting 
UBi. Since Algorithm 3 generates a rigorous upper 
bound, one can terminate the program early even before 
the tightest results are obtained, as long as the intermedi- 
ate results have met the evaluation/design requirements. 

V. Performance / Efficiency Related Issues 

In this section, different schemes to further improve the 
efficiency/performance of UB^ or to confirm the tightness of 
UBi will be discussed, and the end results will be compared 
with those of the Monte-Carlo simulations in Section VII. 

A. Pruning the Tree 

To control the growth rate of the size of T in a manageable 
fashion, we introduce the following two lossless pruning rules. 

Rule 3 — The Equation: • f(y) = 

We first note that • /(y) = for any /(y). Therefore, 
if an inactive variable node v is with its input observation y 
grounded to 0, then we can completely discard all messages 
entering v. Or equivalently, we can prune all subtrees rooted 
at v and the size of T will be reduced. Rule 3 can be applied 
to the right subtree in Fig. 4(b) every time a pivoting is 
performed, since the algorithm deactivates many active nodes 
in the right subtree by hardwiring their observations to 0. See 
Fig. 4 and APPENDIX I for illustration and further reference. 

Rule 4 — The Degenerate Pivoting 

We first note that the following equation holds for all 
iterative decoding functions g(y). 

y k ■ sCy?" W£+i) - yk ■ 5(y^ _1 iyfc+i), (8) 

for all y G {0,1}". 

For any newly-added leaf Vk with free observation yk, define 
yca(ufe) and the corresponding active variable node v' k (of free 
observation yk) as in Algorithm 3. Motivated by the above 
equation, if yca(vk) — v' k , i.e., the active node v' k itself is 
an ancestor of the newly added leaf Vk, then no pivoting is 
necessary and we can directly deactivate Vk by hardwiring its 
free observation yk to 1, as suggested by the right-hand side 
of (8). 

Rule 4 can be viewed as a degenerate case of Rule 2. Sup- 
pose yca(wfe) = v' k and Rule 2 is performed, which generates 
the duplicated subtrees as in Fig. 4(b). Since yca(vfc) = v' k , 



one of the two messages entering the root of the right subtree 
must be yk = 0. By invoking Rule 3, the entire right subtree 
will be pruned and the original pivoting rule collapses to 
Rule 4. 

Whether yca(ufc) = v' k should be checked every time a 
pivoting is about to be performed. If yca(vk) — v' k , then we 
can use Rule 4 instead of pivoting, and the computational cost 
is thus reduced. 

These two rules help the algorithm to use its computational 
resources in a more efficient way, especially the use of 
memory. The pruning rules also play an important role during 
the proof of the optimality of Algorithm 3. 

B. The Leaf-Finding Module 

The tightness of UB^ in Algorithm 3 depends heavily on 
the leaf-finding (LF) module invoked in Line 4. From the 
perspective of narrowing the set of potential stopping sets, the 
LF module determines the reduction from X 4 to X t+ i during 
each iteration. A properly designed LF module is capable of 
resulting in an upper bound UB^ of which the asymptotic order 
is +1 to +3 better than that of a random LF module. To further 
illustrate the importance of the LF module, the ultimate benefit 
of an optimal LF module is stated in the following theorem. 

Theorem 4 (The Optimal LF Module): Following the nota- 
tion in Theorem 3, with an optimal LF module, the simple 
pruning rules in Section V-A, and an infinite amount of 
memory, there exists to < oo such that 

Xd, = X t , Vi > t - 

In words, our narrowing search is able to implicitly identify 
all stopping sets containing Xi through the constructed tree 
T, which is of a special structure that will be used later for 
computing the upper bound UBj. Furthermore, this implicit 
identification can be achieved within a finite number of steps. 

Theorem 4 also demonstrates the importance of a prop- 
erly designed LF module and the pruning rules, namely, if 
a suboptimal LF module is used or the pruning rules are 
not employed, X t may be strictly bounded away from the 
collection of all stopping sets X Xi , as t — > oo, and a huge 
performance loss is expected. The proof of Theorem 4 is 
provided in APPENDIX IV. 

Corollary 1 (Order Tightness of Algorithm 3): When 
combined with an optimal LF module and the pruning rules, 
the UBi computed by Algorithm 3 is tight in terms of the 
asymptotic order if sufficient computational resources are 
provided. Namely, 3C > such that ^$ < C for all 

£G (0,1]. 

Proof: This is a direct result of Theorem 4 and the order 
preserving property of Rule 1, the only relaxation rule. To 
be more explicit, Theorem 4 implies /r,*(y) = fi(y) for all 
t > t ,y G {0,1}™. By Proposition 4 in APPENDIX II, the 
proof is complete. ■ 
For all our experiments, an efficient approximation of the 
optimal LF module, motivated by the proof of Theorem 4, 
is adopted. Running on a personal computer, Algorithm 3 is 
capable of constructing asymptotically tight upper bounds for 
rate- 1/2 LDPC codes of n < 100 and of minimal stopping 
distances 9-11. A composite approach will be introduced later, 
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which further extends the applicable range, and asymptotically 
tight upper bounds can be obtained for codes with n ~ 300- 
500 and minimal stopping distances 11-13. 



sets is provided. To further demonstrate the versatility of the 
proposed algorithm, we discuss the following generalizations, 
including the fc-out trapping set exhaustion algorithms. 



C. Confirming the Tightness of UIV An Exhaustive List of 
Minimum Stopping Sets 

Two important features of the proposed algorithm are the 
easy confirmation of the tightness of UB^ and a byproduct 
tight lower bound LBj, which together with UB^ bracket 
the performance of the parity-check code of interest. In this 
subsection, we will also discuss how to construct an exhaustive 
list of minimal stopping sets containing Xi so that Algorithm 3 
can be used as an algorithm for determining the minimal 
stopping distance from the bit x± perspective. 

To this end, after t iterations of Algorithm 3, we first 
exhaustively enumerate the elements of minimal weight in 
X t . Denote the weight as wr and denote the collection of 
the minimal weight elements as X m ; n C X t . 

Corollary 2 (Tightness Confirmation): If 3x 6 X m j n that 
is a stopping set, then UBj is tight in terms of the asymptotic 
order. Otherwise, UB^ is loose. 

Corollary 3 (Stopping Set Exhaustion): Let X m j n gg C 
X min denote the collection of all elements x 6 X min 
that are also stopping sets. If X m ; n ss is not empty, then 
X m i n .ss exhausts all minimal stopping sets containing bit Xi- 
If X m i n> ss = 0, then there is no stopping set of size < wr- 

Corollary 4 (The Tight Lower Bound): A non-empty 
X m in.ss can be used to derive a lower bound E{/LB,i(y)} 
that is guaranteed to be tight in both the asymptotic order 
and the multiplicity as described in Section IV-A. 

Proofs of Corollaries 2 to 4: We first notice that wr is 
the asymptotic order of both E {/r(y)} and the upper bound 
UBi due to the order-preserving property of Rule 1 . Suppose 
3x £ X m ; n that is a stopping set. By Theorem 3, no stopping 
set containing Xi has weight less than wr and the asymptotic 
order of pi = E{/,(y)} is wr- UB^ is thus an upper bound 
of E {/i(y)} tight in the asymptotic order. For the other case 
that no element of X m ;„ is a stopping set, by Theorem 3, any 
stopping set containing Xi must have weight strictly larger than 
wr and therefore the asymptotic order of E {/i(y)} must be 
strictly larger than wr- UBi is loose and Corollary 2 is proved. 

By Theorem 3, the collection of all stopping sets containing 
Xi is a subset of X t . If 3x £ X m ; n that is a stopping set, 
then X m ; n ,ss is not empty and all minimal stopping sets must 
be elements of X m ; n and X m i nj ss- If X m ; nj ss is empty, any 
stopping set containing X{ must have weight larger than wr- 
Corollary 3 is proved. 

Corollary 4 is then a straightforward result from the ex- 
haustive list of minimal stopping sets containing x^. ■ 

The exhaustive list X m ; n ,ss gives us a richer understanding 
of the code behavior than the Yes/No answer to the decision 
problem SD(H, t, Xi) in Section III-B, the latter of which gives 
only the existence result while the former of which explicitly 
locates all minimal stopping sets involving x^. 

VI. Further Generalizations 

In the previous sections, a tree-based algorithm for upper 
bounding BER performance and exhausting small stopping 



A. Applications on Shortened and Punctured Codes 

Algorithm 3 is originally designed for arbitrary parity-check 
codes but can be easily generalized for codes with punctured 
or shortened bits as follows. 

If we take a closer look at the puncture operation on a 
particular bit Xk, it is equivalent to deterministically assigning 
"erasure" to yk- I.e., it becomes an erasure channel with 
yk = 1 with probability one. Similarly, shortening 4 a bit 
Xk is equivalent to deterministically assigning value to the 
observation yk- Based on this observation, Algorithm 3 can 
be applied to codes with punctured and shortened bits by 
deactivating the newly added leaf v k in Line 5 and hardwiring 
its corresponding yk to either 1 or depending on whether Vk 
is a punctured bit or a shortened bit. The ability of analyzing 
shortened / punctured codes makes the algorithm applicable 
to irregular/regular repeat-accumulate codes as well. 

Not only can the algorithm deal with the punctured and the 
shortened bits, the upper bound \JB i can be computed for the 
setting of parallel independent channels, in which different bits 
Xk may experience different erasure probabilities ek = E {yk}- 
No modification of Algorithm 3 is necessary since Algorithm 3 
does not assume a common erasure probability for different 
bits Xk- 

B. The Composite Approach 

One brute force method of searching for small stopping sets 
is to exhaustively consider all possible candidates of small 
stopping sets and test them one by one to see whether any 
of them is a stopping set, the complexity of which grows 
exponentially for large n. Although completely impractical, 
the divide-and-conquer approach of brute-force search can 
be combined with our tree-based algorithm. A more efficient 
composite approach can thus be obtained as follows. 

We first notice that the expectation E{/j(y)} can be further 
decomposed as 

M 

E{/,(y)} = E P (A)E{.My)|y £ A k }, 

3 = 1 

where the life's are M events partitioning the sample space 
{0, 1}™. For example, we can define a collection of non- 
uniform Ak's by 

Ai = {ye {0,1}" : 2/10 = 0} 

Ai = {ye {0,1}": yio = 1,2/47 = 0} 

A 3 = {ye{o,i}":t/ 10 = 1,2/47 = 1}- 

Since for any fc, fi(y)\ y eA k is simply another finite code 
with many punctured and shortened bits, Algorithm 3 can be 

4 The term "shortening" generally refers to hardwiring some coded bits and 
not sending them through the channel, the effect of which is that both the 
message and the codeword spaces are reduced. In this work, we use the term 
"shorten a bit x k " to stress that bit x k is hardwired to zero and is not sent 
through the channel. 



12 



applied to each fi(y)\ y eA k respectively and different bounds 
UB^fc > E{/j(y)|y G Ak} will be obtained. A composite 
upper bound can then be constructed by 

M 

C-UB, = p (^fc)UBi, fc > E{/,(y)} = Pi . 

k=i 

In the case that Algorithm 3 is used as a stopping set 
exhaustion algorithm, a composite exhaustion algorithm can 
be obtained by assuming the following slight adjustment. 
Consider the event A 2 as an example. Applying Algorithm 3 to 
.fi(y)lye.42 returns an exhaustive list of the minimal stopping 
sets of the corresponding punctured/shortened code in which 
x w is punctured while X47 is shortened. The resulting minimal 
stopping set x however will not include xio since j/10 = 1 
is considered to be a fixed, punctured structure of the code 
instead of a free observation. By taking the union, {xi } U x 
becomes a minimal stopping set of the original code fi(y) 
with y being confined to A 2 . 

/.(y{ I10 }ux) = 1, and y {2;i0 }ux G A 2 - 

In sum, when the minimal stopping set of interest is the union 
of the punctured bits and the conditional minimal stopping set 
x returned by Algorithm 3, the composite approach is able to 
exhaustively search through the partitioned space (Jfcli ^ 
one event at a time. The reduced sample space in each 
event Ak makes the composite approach more efficient when 
compared to the original algorithm. 

With a properly chosen non-uniform partition {Ak} of 50- 
20,000 events, the composite approach generally results in 
upper bounds of the asymptotic order +1 to +3 higher than 
the original UB^, and is capable of exhausting stopping sets 
of sizes +1 to +3 larger than the capability of the original 
algorithm. With the improved efficiency, our algorithm is then 
able to determine the minimal stopping distance and identify 
all minimal stopping sets for many rate- 1/2 LDPC codes of 
practical length n = 512. 

C. BER vs. FER 

Thus far, all our discussions have been based on the bit- 
wise perspective. We either construct an upper bound on 
the BER pi, or exhaust all minimal stopping sets involving 
bit Xi. Upper bounds for the average BER can be easily 
obtained by taking averages over bounds for individual bits. 
An equally interesting problem is to upper bound the FER. 
In this subsection, we briefly discuss how to upper bound the 
FER or to exhaust the minimal stopping sets from the frame 
perspective. 

We again rely on the Boolean expression framework. By 
noticing that the frame error detector is simply the binary 
OR of all individual bit detectors, we have /FER(y) = 
S"=i /i(y)' which is yet another iterative decoding function. 
Since Algorithm 3 is applicable to all iterative decoding 
functions, the upper bound UBfer and the exhaustive list of 
minimal stopping sets can be obtained by direct application 
of Algorithm 3 on ,/FER,(y) instead of the individual /i(y). 
A graphical interpretation can be obtained by introducing an 
auxiliary variable and check node pair (a; ,J/o) sucn that the 



new variable node x is punctured and the new check node y 
is connected to all n+1 variable nodes from x to x n . The FER 
of the original code now equals the BER p of variable node 
xq and can be upper bounded by Algorithm 3. An efficient 
and straightforward partition {Ak}ke{i,....n+i} f° r the FER 
detection /FER(y) is to let 

Vfc G {1, . . . , n}, A k := {y : yf _1 = 0, y k = 1}, 

and An+i := {y : y = 0}. (9) 

All our FER results are based on the above partition or on a 
finer partition derived from the above basic partition. 

Remark: Since the FER depends only on the worst bit 
performance, it is generally easier to construct tight UBfer 
for the FER than for the BER. One might thus question the 
need of bit-wise upper bounds / exhaustive lists of minimal 
stopping sets, which on the other hand, provide detailed 
performance prediction for each individual bit and is of great 
importance during code optimization and analysis [36], [18]. 

D. Exhausting k-Out Trapping Sets 
The Algorithm 

We are now ready to generalize the bit-wise minimal 
stopping set exhaustion algorithm for fc-out trapping sets 
in a frame-wise perspective. We start with converting our 
bit-wise stopping set exhaustion algorithm to its frame-wise 
counterpart. Namely, using the procedures described in the 
previous sections, one is able to obtain an efficient algorithm 
taking inputs H, t, and a list of punctured bits 5 x p = {x p }, and 
outputting (X m ; n , u> m i n ). If the minimal size of the stopping 
sets is within the searchable range t, X m i n is an exhaustive 
list of the frame-wise minimal stopping sets and w m i n is the 
corresponding minimal size. If the stopping distance is beyond 
the searchable range t, then X m ; n = and u> m j n = t. We 
denote this algorithm as the minimal stopping set exhaustion 
SSE(H,£,x p ). The minimal fc-out trapping set exhaustion 
algorithm is then described in Algorithm 4, the correctness 
of which follows from the reduction stated in Algorithm 1 in 
Section III-B. 

The complexity of Algorithm 4, although being polynomial 
with respect to n, grows on the order of O (n fe ), which makes 
the above algorithm less appealing for cases of large k. Using 
the efficient stopping set exhaustion algorithm SSE(H,i,x p ) 
proposed in this paper, meaningful results, i.e., with the search 
range t large enough for short practical codeword length n w 
500, have been reported for the cases of k = 1, 2. 

On the other hand, the dominating error patterns are gen- 
erally those fc-out trapping sets with small k < 4, as noted 
in [6]. In the same paper, it has been shown that for a rate 
51/64, n = 4096 almost-regular code example, more than 55% 
of the FER is contributed by the minimal fc-out trapping sets 
with k = 0,1,2, equivalent to the (10,0), (9,1), (8,2) near- 
codewords respectively [Figure 5, [6]]. Being capable of ex- 
hausting minimal fc-out trapping sets with fc < 2, Algorithm 4 
is a successful first attempt on the inherently NP-complete 

5 The shortening effect can be easily incorporated by removing the corre- 
sponding columns in H and thus will not be considered separately. 
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Algorithm 4 A minimal fc-out trapping set exhaustion al- 
gorithm based on the stopping set exhaustion algorithm 
SSE(H,£,x p ) 
1: Input: H and t. 

2: Initialization: X m i n <— and w m in <— t. 
3: repeat 

4: Based on the Tanner graph, select k edges connect- 
ing to k distinct check nodes and denote them as 

{(Xi 1 , Cjj), . . . , (Xi k , Cj k )}. 

5: if there is no edge between x iy and c Jfc(( for any k' / fc", 
then 

6: Construct a new H' by removing those columns i in H 
simultaneously satisfying (1) i 7^ i n ,Va = l,...,k, and 





(2) 3o € {1, . . . , k} such that ff jal 


= 1. 


7: 


Construct a new H" by removing 


rows jo,Va — 




from H'. 




8: 


Let the punctured bits be x p <— {x 


a :Vo = l,... 


9: 


(Xtemp, Wtcmp) < SSE(H , t, X ? 


)■ 


10: 


if wtcmp + fc < w min then 




11: 


X m in < Xtcmp ° Xpt <Mld W m i n 




12: 


else if Wtemp + k = w m m then 




13: 


Xmin < Xmin U (Xtcmp ° X p ).t 




14: 


end if 




15: 


end if 





16: until all possible selections of k distinct edges are exhausted. 
17: Output: Xmin and w m in- 

f: The "o" operation is defined as follows. Any element in X o x p is of the 
form xUx, for some x € X and vice versa. 



problem of trapping set exhaustion. More numerical examples 
will be reported in Section VII, including the exhaustion of 
all minimal 2-out trapping sets of size 8 for the (155,64,20) 
Tanner code [29]. 

Good codes for BECs are good for other channels 

It is observed in [34] that good codes (in terms of error 
floors) for BECs are generally good for other channels as 
well. As a byproduct of Algorithm 4, Theorem 5 provides 
a justification of this phenomenon as follows. 

Definition 3: A code is "(^S,#cm) uniformly good" for 
BECs if after removing any #c < #cm check nodes, the 
resulting code has minimal stopping distance no less than d$- 

Theorem 5: A (d$, #cm) uniformly good code (for BECs) 
has minimal fc-out trapping distance no less than ds for all 
k < #c M - 

Proof: This is a straightforward result of the correctness 
of Algorithm 4. ■ 
Codes constructed based on girth optimization are generally 
uniformly good for BECs, since removing any check node 
will only increase the girth, which is the lower bound of 
the minimal stopping distance. Theorem 5 guarantees that 
any uniformly good codes for BECs are also good for other 
channels since they have good fc-out trapping distances as well. 

Some Remarks 

As the key building block of trapping set exhaustion, the 
minimal stopping set exhaustion SSE(H, t, x p ) plays a central 
role in the exhaustion of error-prone patterns, including both 
the stopping and trapping sets, and deserves the attention of 
future research along this direction. Two final remarks on the 
efficiency and algorithmic issues are as follows. 



Remark 1: The core of our results is the bit-wise Algo- 
rithm 3. Due to its bit-based characteristics, the efficiency 
of our algorithms can be improved for codes of special 
structure by using the corresponding automorphism, including 
the Margulis codes, the Ramanujan-Margulis codes, and the 
lifted codes based on permutation matrices, etc. Taking the 
cyclically lifted codes with lifting factor L as an example, 
the complexity of Algorithm 4 for fc-out trapping sets can be 
reduced by a factor of 2 k ~ 1 L, which makes our algorithm 
especially suitable for the protograph codes that are based on 
small base codes with high lifting factor L [52]. 

Remark 2: For large k, exhausting fc-out trapping sets 
becomes tricky especially when the minimal variable node 
degree is < fc. If a single variable node v is of degree dcg(u) < 
fc, then {v} itself is the minimal fc-out trapping set and would 
thus dominate the outputs of Algorithm 4 and make the results 
useless. For example, searching minimal 2-out trapping sets of 
an irregular code with degree 2 nodes always returns trapping 
sets containing single degree 2 nodes. A simple remedy is to 
carefully select the {(x il ,Cj 1 ), . . . , (xi k ,Cj k )} of interest in 
Line 4 of Algorithm 4 and preclude the uninteresting combi- 
nations. A more effective method would require unambiguous 
definitions of 'non-trivial" fc-out trapping sets and is currently 
under investigation. 

VII. Numerical Experiments 

The experiments will be divided into four categories: 
(i) non-sparse codes, (ii) LDPC codes with random con- 
struction, (iii) error-floor optimized LDPC codes, and (iv) al- 
gebraically constructed codes. We use SSE and fc-TSE as 
shorthand of the minimal stopping set exhaustion and the 
minimal fc-out trapping exhaustion algorithms discussed in the 
previous sections. 

A. Non-Sparse Codes 

1) The (23,12) Binary Golay Code: The standard parity 
check matrix of the Golay code is described by H = [H' I], 
where I is an 11 x 11 unity matrix and H' is as follows. 

/ 100111000111 \ 
101011011001 
101101101010 
101110110100 
110011101100 
110101110001 
110110011010 
111001010110 
111010100011 
111100001101 

V oiiiiiiiini / 

The minimal stopping distance of the Golay code is 4 and all 
130 minimal stopping sets can be found by the proposed SSE 
in two seconds. All bits are involved in at least one stopping 
set of size 4, and thus their BERs are of similar magnitudes. 
Fig. 5 compares the upper bound (UB), the composite upper 
bound (C-UB), the Monte-Carlo simulation (MC-S), and the 
tight lower bound (LB), for bits 0, 5, and 20. As illustrated, 
C-UB and LB tightly bracket the MC-S results, which shows 
that the UB and C-UB are capable of decoupling even non- 
sparse Tanner graphs with plenty of cycles. However, the non- 
sparsity slows down the efficiency of SSE considerably when 



H = [H' I], H' 
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(a) n = 50 



(b) n = 72 



(c) n = 144 
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TABLE I 

Performance Statistics for Complete Randomly Constructed (3,6) LDPC Codes: "Num. bits" is the number of bits with the specified asymptotic 

order, "order*" is the num. bits with UBs tight only in the order. "+ multi*" is the num. bits with UBs tight both in the order and in the multiplicity, "order >" is the num. bits 

with a UB of the specified order while no bracketing lower bound can be established. 
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V0 MC-S 
V0:U8 
V0 C-UB 
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VF, C-UB 
V5: LB 
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Fig. 5. Comparisons among the upper bound (UB), the composite upper bound (C-UB), the Monte- Carlo simulation (MC-S), and the side product tight 
lower bound (LB) for bits 0, 5, and 20 of the (23,12) binary Golay code. The corresponding asymptotic (order, multiplicity) pairs obtained by UB, C-UB, 
and the actual BER is are V0: {(3, 10), (4,75), (4,75)}, V5: {(3, 15), (4,50), (4,45)}, and V20: {(4,221), (4,27), (4, 1)}. 
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Fig. 6. Comparisons among the UB, the C-UB, the MC-S, and the LB for bits 0, 26, and 19 of a randomly generated (3,6) LDPC code with n = 50. 
The asymptotic (order, multiplicity) pairs of the UB, the C-UB, and the actual BER are V0: {(4, 1), (4, 1), (4, 1)}, V26: {(6, 2), (6, 4), (6, 2)}, and V19: 
{(7, 135), (7, 10), (7, 5)}. 




Fig. 7. Comparisons among the UB, the C-UB, the MC-S, and the LB for bits 41, 25, and 60 of a randomly generated (3,6) LDPC code with n = 
72. The asymptotic (order, multiplicity) pairs of the UB, the C-UB, and the actual BER are V41: {(4, 1), -, (4, 1)}, V25: {(7, 1), -, (7, 1)}, and V60: 
{(7, 19), (8,431), (8, 11)}. 
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compared to sparse LDPC codes of similar sizes, due to the 
higher frequency of invoking the pivoting rule. Nonetheless, 
the proposed SSE still demonstrates superior performance 
when compared to brute force search. For comparison, the 
extremely short codeword length of the Golay code makes the 
stopping set exhaustion problem still within the reach of the 
brute-force search, which requires ( 2 4 3 ) = 490, 314 trials. 

B. LDPC Codes with Random Construction 

1) A (3,6) LDPC Code with n = 50: A (3,6) LDPC code 
with n = 50 is randomly generated, and the UB, the C-UB, the 
MC-S, and the tight LB are performed on bits 0, 26, and 19, as 
plotted in Fig. 6, and the statistics of all 50 bits are provided 
in TABLE 1(a). The UB is tight in the asymptotic order for 
all bits while for 34 bits, the UB is also tight in multiplicity. 
Among the 16 bits for which UB is not tight in multiplicity, 
1 1 bits are within a factor of three of the actual multiplicity, 
which is obtained from the bit-wise exhaustive list of stopping 
sets obtained from the SSE. 

In contrast with the Golay code example, the tight per- 
formance can be attributed to the sparse connectivity of 
the corresponding Tanner graph. The considerably smaller 
size of the resulting tree, which follows from the smaller 
variable and check node degrees and from the less chance of 
invoking the pivoting rule, favors the memory-usage-intensive 
SSE algorithm significantly. As can be seen in Fig. 6(c), the 
composite approach C-UB possesses the greatest advantage 
over simple UBs that are not tight in multiplicity. The C-UB 
and the LB again tightly bracket the asymptotic performance. 

2) A (3,6) LDPC Code with n = 72: The UB, the C-UB, 
the MC-S, and the tight LB are applied to bits 41, 25, and 60, 
as plotted in Fig. 7 and the complete statistics of all 72 bits 
are included in TABLE 1(b). Almost all asymptotic orders and 
most multiplicities can be captured by the UB with only two 
exception bits. Both of the exception bits are of order 8, which 
is computed by the composite SSE for each bit respectively. 

3) A (3,6) LDPC Codes with n = 144: Complete statistics 
of all 144 bits are presented in TABLE 1(c), and we start to 
see many examples (101 out of 144 bits) in which the simple 
UB is not able to capture the asymptotic order and we have to 
resort to the C-UB for tighter results. It is worth noting that 
even the simple UB is able to identify some bits with BER of 
order 9, which requires ( 1 f) w 3.55 x 10 12 trials if a brute 
force search is employed. 6 

C. The Error-Floor Optimized LDPC Codes 

For the randomly constructed LDPC codes in the previous 
examples, the frame-wise minimal stopping distances range 
from 2 to 4, which does not present any challenge for 
the proposed frame-wise SSE. That is the reason why we 
deliberately omit the discussion of the frame-wise minimal 
stopping set exhaustion in the previous examples and focus 
mainly on using the SSE as a bit-wise exhaustion or a bit- 
wise upper bound. For the following, we consider the regular 

For the bit-wise brute force search, since the target bit is always included, 
we need only to search through ( 1 g 3 ) possibilities instead of ( 1 g 4 )- 
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Fig. 8. Comparisons among three different regular (3,6) codes in the 
following order. A short n = 64 code with minimal stopping distance fig 
optimized to 8 and multiplicity 89 confirmed by the stopping set exhaustion. A 
randomly chosen n = 512 code with d$ = 11 and multiplicity 1 determined 
by the stopping set exhaustion. An optimized n = 512 code with rig > 13. 
For the last code, one stopping set of size 24 is identified during the Monte- 
Carlo simulation. 

and irregular codes optimized for the error-floor performance 
and having minimal stopping distances 8-13. The SSE will 
be used as the exhaustion algorithm and to provide exhaustive 
lists of minimal stopping sets. The trapping set exhaustion 
algorithm /c-TSE will be applied to different codes of length 

500 as well. This range of codeword lengths is large enough 
to be of great practical importance and is beyond the reach of 
any brute-force search algorithm. On the other hand, it is still 
short enough for the proposed SSE and /c-TSE algorithms to 
construct exhaustive lists of dominating error-prone patterns. 

The codes discussed herein are optimized by the code an- 
nealing method, presented in a companion paper [18], that uses 
the exhaustive lists of error-prone patterns generated by SSE 
and /c-TSE as the objective functions for code optimization. 
For all the Monte-Carlo simulation results, 100 error events 
are observed for each simulation point. 

Regular (3,6) LDPC Codes on BECs 

Experiment 1: An optimized (3,6) LDPC code with n = 64. 
Using the exhaustive list as the objective function, its minimal 
stopping distance has been optimized to 8. Even for codes of 
this small size, the brute-force search requires ( 6 g 4 ) w 4.43 x 
10 9 computations, which can hardly be achieved by software 
simulations. All 89 minimal stopping sets are identified by 
SSE, and the exhaustive list provides a tight lower bound 89 x 
e 8 for the FER. The corresponding Monte-Carlo simulation 
results and the tight lower bound are plotted in Fig. 8. For 
comparison, the girth for any (3,6) code of length n — 64 is 
upper bounded by 6, which counts both variable and check 
nodes, while the minimal stopping set of this code consists 
of 8 variable nodes. This is a concrete example showing that 
even with a limited girth size (3 variable nodes), one can still 
construct codes with much larger minimal stopping distances. 

Experiment 2: A randomly chosen (3,6) LDPC code with 
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Fig. 9. Comparisons among three different irregular (Ai (x), pi(x)) codes 
with their minimal stopping distances rig ar, d multiplicity mg determined by 
the minimal stopping set exhaustion. A short n = 72 code with ds optimized 
to 8 and mj = 71. A randomly chosen n = 576 code, denoted by "Rand," 
with d<j = 2 and mg = 2 has the poorest FER performance due to the 
existence of small stopping sets. In contrast with "Rand," the error floor of the 
optimized n = 576 code with d$ = 13 and mg = 104, denoted by "Opt," 
is dramatically improved over its random counterpart. Tight lower bounds 
derived from the exhaustive lists of minimal stopping sets are plotted for 
comparison. 

n = 512. It is well-known that regular (3,6) codes generally 
have superior error-floor performance with random construc- 
tion, which is evidenced by this example. We arbitrarily choose 
one realization from the random code ensemble. This n = 512 
(3,6) code has minimal stopping distance ds = 11 with 
multiplicity 1. The SSE is able to identify the only one 
minimal stopping set, and the resulting tight lower bound 
coincides with Monte-Carlo simulation when e < 0.3; see 
Fig. 8. Using Monte-Carlo simulation, one can also identify 
the same minimal stopping set, but cannot claim with 100% 
certainty that there is no other stopping sets of size 1 1 , which 
is a major distinction between the proposed SSE algorithm and 
any randomized enumeration algorithm, e.g. the error impulse 
methods. 

Experiment 3: An optimized (3,6) LDPC Code with n — 
512. An optimized (3,6) code with n = 512 is constructed 
by using the exhaustive list of stopping sets as the objective 
function. All candidates of sizes < 12 are exhausted and none 
of them is a stopping set. The minimal stopping distance must 
be > 13. One stopping set of size 24 is observed during the 
Monte-Carlo simulation, which gives us an upper bound on the 
actual minimal stopping distance. As can be seen in Fig. 8, 
for this optimized code with minimal stopping distance > 13, 
no error floor can be observed by software simulations until 
FER = 10" 7 . 

Irregular LDPC Codes on BECs 

A more interesting subject is irregular LDPC codes with 
degree distributions optimized for threshold performance [11], 
which turns out to be the most computationally friendly 
instance for the proposed SSE algorithm. For rate- 1/2 irregular 
codes of length 512, the searchable range t can be extended to 



13, which is +2 better than its regular code counterpart. Using 
the conventional polynomial representation of degree distri- 
butions [11], we consider codes of the following optimized 
variable and check node degree distributions: 

Xi(x) = 0.41667a; + 0.16667a; 2 + 0.41667a; 5 
px(x) = x 5 , 

which has asymptotic erasure probability threshold e* w 
0.4775 [53]. Codes of different codeword lengths and con- 
struction methods are discussed as follows and their results are 
illustrated in Fig. 9. For comparison, to exhaust all stopping 
sets of size 13 for a n = 576 code requires f 576 ) rj 1.08 x 10 26 
trials if a brute force approach is adopted. 

Experiment 4: An optimized n — 72 irregular LDPC code. 
Using the above (Ai, p\) degree distributions, an irregular code 
is constructed with optimized minimal stopping distance ds = 
8 as described in [18], and all of the 71 minimal stopping 
sets can be identified. Both codes in Experiments 1 and 4 are 
of minimal stopping distance 8 and of similar sizes, which 
show that although there are inherently many "bad" irregular 
codes, the error floor of "good" irregular codes is comparable 
to that of good regular codes, provided proper optimization is 
performed using the exhaustive list of minimal stopping sets 
as the objective functions. 

Experiment 5: An optimized n — 576 irregular LDPC code. 
Using the same (X\,pi) degree distributions with n = 576, 
one can construct an irregular code with minimal stopping 
distance 13 [18]. All 104 minimal stopping sets have been 
identified by the SSE algorithm and a tight lower bound is 
provided. For comparison, we also plot the FER curve of a 
typical irregular code of the same degree distributions (Ai, pi). 
At e = 0.3, an improvement of a factor 10,000 is reported 
when compared to the typical code performance, which again 
demonstrates the benefit of directly using the stopping set 
exhaustion list as the objective function. 

Irregular LDPC Codes on Gaussian Channels 

We focus our trapping set discussions on binary-input addi- 
tive white Gaussian noise channels (BiAWGNCs). Namely, the 
observation vector y = (2x — 1) + crn, where 1 is an all-one 
vector, n is a standard Gaussian vector with unity covariance 
matrix, and 1/er 2 is the signal to noise (power) ratio. Only 
irregular LDPC codes will be considered and their common 
degree distributions, optimized for threshold performance, are 
as follows: 

A 2 (x) = 0.31961a; + 0.27603a; 2 + 0.01453a; 5 + 0.38983a; 6 
p 2 (x) = 0.50847a; 5 + 0.49153a; 6 , 

with an asymptotic threshold a* w 0.937 [53]. All the minimal 
stopping sets or fc-out trapping sets are identified by the SSE 
and fc-TSE discussed previously. The corresponding Monte- 
Carlo simulation results are in Fig. 10. 

Experiment 6: "Rand" is an n = 512 code arbitrarily chosen 
from the (\2,p2) irregular code ensemble with an additional 
constraint that there is no multiple edge in the corresponding 
bipartite Tanner graph. Without any graph optimization, the 
performance is not impressive for irregular codes of this short 
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Fig. 10. Monte-Carlo simulations for irregular (\2(x), p2(x)), n = 512 
codes with different constructions: "Rand," "SS Opt," and "TS+SS Opt" 
as described in Experiments 6, 7, and 8 respectively. Their corresponding 
(min_size,multiplicity) of the fc-out trapping sets for fc = 0, 1 respectively 
are: "Rand" (2, 1), (2, 8); "SS Opt" (13, 40), (5, 4); "TS+SS Opt" (11, 12), 
(10, 24). 80 decoding iterations are performed and 100 error events are 
observed for each simulation point. 



length. To further pinpoint the cause of the bad performance, 
its minimal fc-out trapping sets are exhausted for fc = 0, 1 
(the 0-out trapping sets correspond to the stopping sets), 
and their (min_size,multiplicity) pairs are (2,1) and (2,8) 
respectively. Its short minimal trapping distances explain the 
poor performance of this code. 

Experiment 7: "SS Opt" is an n = 512 code with degree 
distributions (A2,P2) but optimized for maximizing the mini- 
mal stopping distance by the code annealing method in [18]. 
Its corresponding (min_size,multiplicity) of the fc-out trapping 
sets for k = 0, 1 are (13,40) and (5,4) respectively. This 
example confirms the phenomenon that codes optimized for 
BECs are generally good for other channels as well since 
the minimal 1-out trapping distance has been improved to 5 
as a byproduct of optimizing the minimal stopping distance. 
This correlation between the BEC performance and the per- 
formance on other channel models has been used to develop 
several design heuristics for general channels, including partial 
elimination of stopping sets [36]. 

Experiment 8: "TS+SS Opt" is an n — 512 code with degree 
distributions (\2,P2) but optimized for maximizing both the 
minimal 1-out trapping distance and the minimal stopping 
distance (the minimal 0-out trapping distance) by the code 
annealing method. Its corresponding (min_size,multiplicity) of 
the fc-out trapping sets for fc = 0, 1 are (11, 12) and (10,24) 
respectively. Namely, for any set of contaminated bits of size 
< 9, there are at least two extrinsic messages that might break 
this error pattern. The BEC performance of "TS+SS Opt" is 
not as good as "SS Opt," since the minimal stopping distance 
is reduced from 13 to 11. However, the minimal 1-out trapping 
distance optimization further brings down the error floor 
for the non-erasure BiAWGNCs. No error floor is observed 
until FER=10 -6 . The example demonstrates the benefit of 
using direct objective functions during code optimization in 



a channel-by-channel basis. The minimal trapping distance 
optimization is feasible due to the efficient minimal trapping 
set exhaustion Algorithm 4. 

Remark 1: We did not exhaust the minimal 2-out trapping 
sets since there are many degree 2 nodes in these codes and 
each of them is by definition a 2-out trapping set. 

Remark 2: When applied to codes of larger sizes, e.g., 
n > 1000, the searchable range t of the SSE and fc-TSE 
also increases when compared to n w 500 codes due to the 
larger girth and the more tree-like structure for longer codes. 
Nevertheless, the minimal stopping / trapping distances for 
longer codes generally grow faster than the searchable range 
t and fewer tight results can be obtained for the error-floor 
optimized regular/irregular codes of length n > 1000. 

D. Algebraically Constructed Codes 

We apply the exhaustion algorithms to the following 
well-studied codes: the (155,64,20) Tanner code [29], the 
Ramanujan-Margulis (2184,1092) code with q = 13 and p = 5 
[46], and the (672,336) Margulis code with p = 7 [47]. 
Since all these codes are regular codes of minimal variable 
node degree 3, the 2-out trapping set exhaustion is performed 
in addition to the stopping set and the 1-out trapping set 
exhaustion. 

1) The (155,64,20) Tanner Code: The minimal Hamming 
distance of this code is known to be 20, which is computed by 
MAGMA, a software suite taking advantage of the correspond- 
ing algebraic structure [54]. One can easily locate a stopping 
set of size 18 by Monte-Carlo simulation, which serves as 
an upper bound of the minimal stopping distance. The SSE 
and fc-TSE are then performed to gather new results regarding 
different types of error-prone patterns of this code. 

For stopping sets (0-out trapping sets), all candidates of size 
< 12 have been exhausted and none of them is a stopping set, 
which results in a lower bound 13 on the minimal stopping 
distance. For 1-out trapping sets, all candidates of size < 11 
have been exhausted and none of them is a 1-out trapping set, 
which results a lower bound 12 on the minimal 1-out trapping 
distance. For 2-out trapping sets, we are able to exhaustively 
locate all 465 minimal 2-out trapping sets, which are of size 8. 
All 465 minimal 2-out trapping sets can be obtained from the 
following 5 representatives by the automorphisms discussed 
in [29]. 

7,17,19,33,66,76,128,140 
7,31,33,37,44,65,100,120 
1,19,63,66,105,118,121,140 
44,61,65,73,87,98,137,146 
31,32,37,94,100,142,147, 148. 

Recently, the instanton analysis in [16], [9] identified some 
dominating error patterns, termed as instantons in their paper. 
Each of the reported instantons contains one minimal 2-out 
trapping set as a substructure. 7 These results again confirm 

7 In [16], [9], the constituting permutation matrices are cyclically shifted 
to the right, which results in different indexing than the original construction 
in which the permutation matrices are cyclically shifted to the left [29]. An 
index remapping is necessary for direct comparison. 
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that the dominating error patterns are generally fc-out trapping 
sets with small fc. 

2) The Ramanujan-Margulis (2184,1092) Code w. q = 
13, p = 5: MacKay et al. [7] first pointed out there is a 
codeword of Hamming weight 14 for the Ramanujan-Margulis 
(2184,1092) code with q = 13, p = 5. By the automorphism 
of the code, there are at least 1,092 codewords of such 
weight. By a refinement of the error impulse method, Hu 
et al. [32] also identified the same 1,092 codewords for 
the Ramanujan-Margulis (2184,1092) code. The remaining 
question is whether there is any other codeword of equal or 
smaller Hamming weight. There are at least two approaches to 
answering this question: one is to use the algebraic structure of 
the Ramanujan-Margulis code and mathematically prove that 
there is no other codeword of equal or smaller weights. The 
second approach is by the exhaustive search, which requires 
6.2 x 10 35 trials if by a brute force search but is feasible by 
the SSE algorithms. 

Our stopping set exhaustion algorithm shows that there are 
"only" 1,092 stopping sets of size 14 and there is no stopping 
set of smaller size. Since any codeword must be a stopping set, 
the minimal Hamming distance of the Ramanujan-Margulis 
(2184,1092) code must be 14 and its multiplicity must be 
1,092. What MacKay et al. and Hu et al. found are indeed 
the minimal codewords. 

We also exhaust its fc-out trapping sets for k = 1,2 
respectively. All candidates of 1-out trapping sets of size < 12 
and all candidates of 2-out trapping sets of size < 9 are 
exhausted and none of them is a trapping set, which results 
in lower bounds of 13 and 10 on the minimal 1-out and 2- 
out trapping distances, respectively. The lack of trapping sets 
of small sizes explains why the dominating error event of 
the Ramanujan-Margulis (2184,1092) code concerns the small 
codewords instead of the small near-codewords. 

3) The (672,336) Margulis code with p — 7 [47]: It is 
known that this code has a codeword of length 16 [9]. The 
SSE and fc-TSE are again performed to gather new results 
regarding the error-prone patterns of this code. 

For stopping sets, the SSE shows that there is no stopping 
set of size < 13. For fc-out trapping sets, there is no 1-out 
trapping set of size < 12 and there is no 2-out trapping of 
size < 9. Since no exhaustive list of any type of minimal 
error-patterns can be obtained, our results serve only as lower 
bounds on the sizes of minimal error-prone patterns. For 
comparison, all stopping sets observed by the Monte-Carlo 
simulation are of size > 16 and no tighter upper bound can 
be obtained for the minimal stopping distance. Combined with 
our results, the minimal stopping and the Hamming distances 
of the (672,336) Margulis code are between 14-16. 

Results for the three algebraically constructed codes are 
summarized in TABLE II. 

Note: The SSE and fc-TSE are directly applied to these 
codes without taking advantage of their algebraic structure 
except for the automorphism, which is used only in the 
statement that results of the bit-wise search is sufficient to 
determine the corresponding results for the entire frame. The 
frame-wise search can thus be simplified to the more efficient 
bit-wise search. 
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Summary of bounding the minimal Hamming, stopping, 1-out trapping, and 
2-out trapping distances for the Tanner (155,64,20) code, the 
Ramanujan-Margulis (2184,1092) code with q = 13, p = 5, and the 
Margulis (672,236) code with p = 7. The pair (d, m) representing the 
minimal distance d (or the range of the minimal distance) and how many 
codewords/stopping sets/trapping sets are of weight d. Upper bounds are 
obtained from enumerations, while lower bounds are from the exhaustion 
algorithm provided herein, f is obtained using computer search MAGMA 

[29]. 



VIII. Conclusion & Future Research 

The problem of exhausting error-prone patterns for arbitrary 
LDPC codes with finite length has been thoroughly discussed, 
and efficient exhaustion algorithms have been devised for 
obtaining both the minimal stopping sets for binary erasure 
channels and the minimal trapping sets for general channels. 
The algorithms, based on the decoding tree of the iterative 
decoder, are equivalent to a narrowing search of potential 
error prone patterns. The optimality of the algorithms has 
been proven and several improvements have been made to 
further improve the efficiency, including a composite approach 
combining the divide-and-conquer strategy of the brute-force 
search and the proposed algorithm. All the proofs and deriva- 
tions are based on the proposed Boolean expression framework 
for iterative decoding functions over binary erasure channels. 
Extensive numerical experiments have been conducted on 
different codes, including randomly constructed codes, error- 
floor optimized codes, and algebraically constructed codes. For 
rate- 1/2 codes of short practical lengths n « 500, both the 
minimal stopping sets and the minimal trapping sets can be 
listed exhaustively. The exhaustive list of error-prone patterns 
finds applications in code behavior analysis, upper and lower 
bounding code performance, and finite code optimization for 
improved error- floor performance. 

The NP-completeness of the trapping set exhaustion al- 
gorithm has also been established, which demonstrates the 
inherent hardness of this problem. Implying that there is little 
chance that an efficient algorithm for exhaustively enumerating 
small stopping / trapping sets exists, the NP-completeness 
argument is genuinely an asymptotic worst-case analysis, 
which does not preclude the existence of efficient algorithms 
for codes of short lengths. The algorithms discussed in this 
work serve as the first successful step toward addressing 
the seemingly intractable problem of exhausting error-prone 
patterns for LDPC codes of practical lengths. 

There are many possible approaches to enhancing the ef- 
ficiency of the proposed algorithms or to designing better 
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algorithms for even longer codes. We conclude this paper 
by providing a short, non-comprehensive list of potential 
improvements. 

1) Several parts of the existing algorithms can be further 
improved, including but not limited to a better design 
of the leaf-finding module, as discussed previously, and 
the partitioning events for the composite approach. The 
effects of different partitioning events on the efficiency 
of the algorithms can easily be as large as a factor 
of 100. With considerable amount of design freedom 
over the partitioning events, we believe the full power 
of the composite approach is not utilized. One possible 
direction is to use the information from the Monte-Carlo 
simulation as a design guideline for partitioning events. 

2) The efficiency of the current algorithms is sufficient for 
codes of short lengths n w 500. Better implementations 
with different data structures, machine-optimized soft- 
ware codes, or even a hardware implementation of the 
algorithms could further improve the applicable range 
to n w 1000 or larger, which will have even broader 
impact on finite length LDPC code analysis and design. 

3) The minimal trapping sets are the error-floor determining 
factor for general channels, and the sizes of interest 
are generally less than 15, as opposed to the minimal 
stopping distances of interest, which are generally 15- 
30. The above reasons make the exhaustion of trapping 
sets an especially appealing problem due to its practical 
importance and its much smaller search range. In this 
work, the minimal trapping set exhaustion problem is 
solved by the reduction to the minimal stopping set 
exhaustion, and significant efficiency loss is incurred. 
Directly designing algorithms for the minimal trapping 
set exhaustion should recover the performance loss and 
extend the applications to codes of medium length n w 
1000. 

Appendix I 
Detailed Descriptions of Algorithm 3 

Some technical details of Algorithm 3 are discussed herein. 

How to grow a leaf from the Tanner graph? 

Since every time a variable node Vk is added to T, all its 
immediate check node children are added to T as well, 8 a 
"leaf in T actually refers to a small sub-tree rooted at Vk, 
consisting of Vk and all its immediate children cj. Therefore, 
all possible "positions" of growing "leaves" are check nodes. 
Suppose we would like to grow another leaf at check node 
cj . From Cj 's immediate parent v io , one can look up the 
directed (cj , v io ) edge in the Tanner graph, and consider its 
incoming extrinsic message flows (vi,Cj ) such that i ^ i . 
If one Vi is chosen and added to T through Cj , then the 
immediate children of the newly-added Vi refer to those Cj ^ 
Cj connected to u f in the Tanner graph. There are cases in 
which some Vi's have already been added to Cj while some 
Vi's have not. Then the next leaf to grow has to be chosen 

8 The only exception is during the finishing step Line 16. 



from those v^s that have not been previously added to check 
node cj . 

Active and Inactive Variable Nodes 

A special notation in Algorithm 3 is that each variable 
node Vk has been assigned a status either active or inactive. 
The reasons behind this notation are as follows. Unlike the 
Tanner graph, in which each variable node appears exactly 
once, in the infinite-sized decoding tree, the same variable 
node may show up infinitely many times. Therefore, the term 
"a variable node Vk" has two different meanings in that it either 
refers to "the node" in the decoding tree T associated with 
an AND operation or refers to the corresponding "variable" 
in the Tanner graph accepting observation yk. The status of 
a variable node is to emphasize the link between the free 
observation yk in the Tanner graph and a node located in the 
decoding tree. Each active node serves two purposes: taking a 
free observation y as its input and acting as an AND operation 
in the iterative decoding function. For an inactive node, no 
free input observation is taken and it acts only as an AND 
operation. 

Note: Although originally not associated with any v& in the 
Tanner graph, an auxiliary variable node can still be active 
and has free observation yk as illustrated in the root of the 
left subtree in Fig. 4(b). Those auxiliary nodes have to be 
considered in Line 7 and therefore, the node v' k may be an 
auxiliary tree node and have no direct connection to any node 
in the Tanner graph. 

yca(vfc) and v' k 

For any two nodes in a tree, their youngest common ancestor 
can be uniquely defined by finding all the common ancestors 
of this pair of nodes and then selecting the youngest of them. 
For any existing active node with free observation yk, we can 
uniquely determine the youngest common ancestor between 
the newly-added leaf and the existing node. In Line 8, yca(vk) 
refers to the youngest among the set of all youngest common 
ancestors generated by different pairs of Vk versus existing 
active variable nodes, and v' k is the corresponding active node 
with free observation y^ resulting yca(iifc) jointly with As 
observant readers might notice, yca(wfc) is unique but v' k may 
not be. Algorithm 3 works for any choice of v' k . 

The pivoting rule depicted in Fig. 4 relies on the assump- 
tion that among all messages coming from the children of 
yca(ufc), there are exactly two messages, denoted by g(y) 
and h(y), sharing a single common determining variable yk- 
This assumption will be proved as a byproduct of proving the 
correctness of Algorithm 3. 

Converting a tree g(y) to its conditional version 
5(yo~Vy£+i) M a fixed y k E {0, 1}. 

Any subtree corresponding to a function g(y) can be 
converted to a conditional subtree <?(yo _1 Oy£ +1 ) as follows. 
Locate all active variable nodes having free observation yk- 
Change their status to inactive by hardwiring their input 
observation yk to 0. Although in Fig. 4(a) only a single square 
is used to represent yk, the conditional function ff(yo _1 Oyfc + i) 
can be obtained only by hardwiring all free observations yk 
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in the corresponding subtree since yk is generally involved 
in more than one product form. The subtree conditioned on 
jjk = 1 can be obtained similarly. 

Evaluating UB,; 

The evaluation of UB^ proceeds iteratively based on the 
formulas in Rule 0. For the case in which an observation yk is 
hardwired to zero/one, we substitute E {yk} — or E {yk} = 1 
into the formula of the variable node function E {f v {y)}- Note: 
the formulas in Rule become pure arithmetic operations 
computing an upper bound of the expected value and lose 
their interpretations as computing the expectations. A more 
accurate notation for the iterative upper bound computation 
should be 

UB E{/„(y)} =UB E{ff(y)} UB EWy)} 
UB E{/c(y)} =UB E{3(y)} +UB E{ , l(y)} 
- UB E{9(y)} UB E{ , l(y)} , 

instead of using the expectation operator. 

Appendix II 
A Proof of The Correctness of Algorithm 3 

The proof of the correctness of Algorithm 3 consists of the 
following two steps. For any T constructed by Algorithm 3 
after t iterations, let fr,t (y) denote the corresponding iterative 
decoding function. We will show that (i) E{/r,t(y)} is no 
larger than the UB^ computed by Algorithm 3, and (ii) the 
decoding function ,fi(y) < fr,t(y) for all y. Since (ii) is a 
restatement of Theorem 3, we will prove (i) in this appendix by 
the following two propositions and leave (ii) to APPENDIX III. 

Proposition 3: In the tree T generated by Algorithm 3, all 
messages entering the same variable nodes must be indepen- 
dent. Namely, if g(y) and h(y) enter the same variable node, 
then they share no common determining variable. 

Proof: An equivalent statement of the above proposition 
is that the youngest common ancestor of any two active 
variable nodes with the same free observation must be a check 
node. A simple proof of the equivalence is as follows. Suppose 
there exist two distinct, dependent messages g(y) and h(y) 
entering the same variable node v. Then g(y) and h(y) share 
at least one common determining variable, denoted by yk, and 
there must be active variable nodes with free observation yk in 
each of the subtrees of g(y) and h(y). Pick one active variable 
node from each subtree. The youngest common ancestor of the 
selected pair of active variable nodes is thus variable node v, 
which contradicts the assumption that the youngest common 
ancestor of any pair of active variable nodes with the same 
free observation is always a check node. The equivalence is 
obtained. 

We then prove by induction that all youngest common an- 
cestors, if they exist, are check nodes. During the initialization, 
there is only one active variable Vi with free observation yi 
in T, and the above statement obviously holds. Suppose the 
statement holds after t iterations of the REPEAT-UNTIL loop 
in Line 3. For the (t + l)-th iteration of adding Vk into T, 
consider two cases: yca(ufc), the youngest of all common 
ancestors, is a check node or not. Using the same notation as in 



Algorithm 3, we use v' k to denote the existing active variable 
node resulting in yca(vfc) jointly with Vk- For the case in which 
yca(ufc) is a check node, no pivoting is involved so one needs 
only to show that after adding Vk there is no other active node 
v'l with free observation yk such that the youngest common 
ancestor of Vk and v' k is a variable node v. We prove this 
statement again by contradiction. Suppose there exists such a 
v k . Since yca(vk) is the youngest of all common ancestors 
and is a check node, yca(wfc) must be a strict descendent of 
v. Therefore, the youngest common ancestor between v' k and 
v' k ' must also be v, which contradicts the assumption that after 
t-iterations all youngest common ancestors of pairs of active 
variable nodes with the same free observation are check nodes. 

Before proving the case in which yca(ufe) is a variable node, 
we first show that if yca(vfc) is a variable node, before piv- 
oting, there are exactly two messages g(y) and h(y) entering 
yca(ufc) such that they share a single common determining 
variable yk- A short argument is as follows. Since yca(vk) 
is a youngest common ancestor of nodes Vk and v' k , there 
must be two completely disjoint paths from yca(vk) to Vk 
and v' k respectively, which implies that Vk and v k must be 
attributed to different messages entering yca(wfe). Therefore, 
there are at least two incoming messages having yk as their 
determining variables. Suppose there is another message g'(y) 
entering yca(vk) which also has yk as its determining variable. 
Then there must be an active variable node v' k in the subtree 
corresponding to g'{y) that also accepts free observation yk. 
The youngest common ancestor of nodes v' k and v k must be 
yca(ufe), which contradicts the assumption that all youngest 
common ancestors must be check nodes after t-iterations. 

To show that after pivoting (i.e., when yca(vk) is a variable 
node) the same statement holds, we first note that after 
pivoting, all active nodes in the subtree rooted at yca(vk) with 
free observation yk are deactivated, with the only exception 
being the root of the left subtree, see Fig. 4 for illustration. 
The active variable node Vk is thus referring to two different 
nodes before and after pivoting. Before pivoting, Vk refers to 
the newly added leaf. After pivoting, Vk refers to the root of 
the left subtree. Suppose after pivoting there exists another 
active variable node v k with free observation yk such that 
the youngest common ancestor of Vk and v k is a variable 
node v. As shown previously, before pivoting, there are exactly 
two messages g(y) and h(y) having yk as their determining 
variables. So before pivoting, all active variable nodes in the 
subtree rooted at yca(vk) with free observation yk are within 
the only two subtrees corresponding to g(y) and h(y). After 
pivoting, all those active nodes will be deactivated with the 
only exception being Vk- Therefore, v k , as an active variable 
node after pivoting, cannot be a descendent of yca(vk), which 
implies that v must be a strict ancestor of yca(ufc). 

The fact that after pivoting v k is not a descendent of yca(wfc) 
and d is a strict ancestor of yca(vk) implies that before 
pivoting, the youngest common ancestor of active nodes v k 
and v' k is also v, which contradicts the assumption that after 
t-iterations all youngest common ancestors of pairs of active 
variable nodes with the same free observation are check nodes. 

Another possibility is that after pivoting with respect to 
Vk, there exist two active nodes v x and v' x , both of whom 
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are accepting free observation y x ^ y^ and the youngest 
common ancestor of this pair is a variable node. We consider 
the following four cases regarding to the tree configuration 
after pivoting: (a) neither v x nor v' x is a descendent of yca(«fc), 
(b) one of them, say v x , is a descendent of yca(ufc) while 
the other is not, (c) both of them are in the same left (or 
right) subtree, and (d) v x and v' x are in the left and the 
right subtrees respectively. For case (a), since the pivoting rule 
only affects the descendents of yca(v/c), v x and v' x are intact 
during pivoting and their youngest common ancestor must be a 
check node due to the induction assumption. For case (b), the 
youngest common ancestor of v x and v' x after pivoting must 
be an ancestor of yca(vfc) since v x is a descendent of yca(wfc) 
but v' x is not. Since v x must be obtained from an active 
variable node during the duplicating step of pivoting, we use 
v' x to denote the original active variable node before pivoting. 
Then the youngest common ancestors between (v x ,v' x ) and 
(v' x ,v' x ) coincide and must be a check node. For case (c), 
since v x and v' x are in the same left (or right) subtree, their 
youngest common ancestor must also be in the same left (or 
right) subtree. As a result, the relative locations among v x , v' x , 
and their ancestor must be preserved during pivoting. Since 
before pivoting the youngest common ancestor between any 
two active nodes accepting free observation y x must be a 
check node, the youngest common ancestor of v x and v' x must 
be a check node as well. For case (d), the youngest common 
ancestor of v x and v' x is the check node of degree 2 taking 
the two subtrees as its children. 

From the above reasoning, after (t + 1) iterations, all 
youngest common ancestors of pairs of active variable nodes 
with the same free observation are check nodes. Proposition 3 
is thus proved by induction. ■ 

Proposition 4: The UB; computed in Algorithm 3 is an 
upper bound on E {,/V,*(y)}- Furthermore, UB^ is of the same 
asymptotic order as that of E {/r.t(y)}- 

Proof: By Proposition 3, all messages entering the 
variable nodes are independent, and one can use Rule to 
compute the exact expectation of the outgoing message from 
a variable node. On the other hand, Rule 1 gives us an upper 
bound on the expectation of the outgoing message from a 
check node, namely, 

E{/ c (y)}<UB E{/c(y)} 

= E{g(y)} + E{h(y)}-E{g(y)}E{h(y)}. 

Using Rule for E {/„(y)} and Rule 1 for E {,f c (y)}, we are 
able to compute their upper bounds assuming we know the 
expectations of all input messages, which are unfortunately 
not available and are the quantities to be bounded. Fortunately, 
both Rule and Rule 1 are monotonically increasing functions 
with respect to the input expectations, and we can thus 
substitute all input expectations with their upper bounds and 
obtain the iterative upper bounding formula: 

UB E {/„( y )} =UB E{3(y)} UB E{/l(y)} 
UB E{/c(y)} =UB E{g(y)} +UB E{ , l(y)} 
- UB E{9(y)} UB E{ , l(y)} , 



which is as if we are computing the expectation value by 
blindly assuming all messages are independent. Since the only 
inequality involved is resulted from Rule 1, which induces no 
order loss, the UBi is an upper bound of E {/r,t(y)} tight in 
the asymptotic order. ■ 

Appendix III 
The Proof of Theorem 3 

Since the collection of all stopping sets involving bit Xi 
can be rewritten as {x C {x\, . . . ,x n } : /i(y x ) = 1} 
where f(y) is the decoder for bit Xi, proving Theorem 3 is 
equivalent to showing that / 4 (y) < /r,*+i(y) < /r,t(y) for 
all y e {0,1}", t e N. To better describe the tree-revealing 
argument used in the proof, we introduce the following lemma 
and notation. 

For any infinite-sized tree T corresponding to an iterative 
decoding function fr{y), a localized subtree of T is defined 
as a subtree U sharing the same root as T. The function fu (y) 
is computed by assuming all edges/messages outside of U 
are hardwired to 1. Namely, the iterative decoding fu(y) is 
performed only within the localized subtree U. By the channel 
degradation argument, we have the following self-explanatory 
lemma. 

Lemma 2: If U is a localized subtree of T, then fu (y) < 
.My) for all y G {0, 1}". 

Proof of Theorem 3: We use T Xi to denote the infinite- 
sized tree corresponding to the iterative decoder fi(y), and use 
T to denote the finite tree T of Algorithm 3 after t iterations 
of the REPEAT-UNTIL loop where t G {0,1,...}. We also 
define To = T Xi to be the first entry of a series of infinite- 
sized trees. By noting that %, containing Vi and its immediate 
cheeck node children, is a localized subtree of T and by 
Lemma 2, we have 

/•r,o(y) = /r (y) 

> /r (y) = h Xi (y) = /i(y), vy e {o, i} n . 

We then consider the situation after the first iteration, 
which naturally consists of two cases corresponding to whether 
pivoting is performed or not during the first iteration. For the 
simpler case, in which yca(wfe) is a check node and no pivoting 
is performed, define T\ = To as the second entry of the series 
of infinite-sized trees. By observing that T is obtained from 
T by adding another leaf that was previously in T but not 
in T , we have that T\ is a localized subtree of T\ = T , and 
To is a localized subtree of T\. Therefore, 

/r,o(y) = /r (y) 

> /r,i(y) = /r 1 (y) 

> hSy) = /r (y) = .My), v y e {o, (io) 

The more interesting case is when a pivoting is performed. 
Let T1/2 denote the tree after adding another leaf but before 
pivoting. By Lemma 2, we have /r,o(y) = /r (y) > 
/r 1/2 (y). Since what pivoting does is to transform the original 
tree T/2 into its equivalent form T, we have /r,i(y) = 
/ri(y) = /r 1/2 (y) for all possible y. Furthermore, the same 
pivoting operation can be applied to the infinite-sized T 
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and we denote the end result as T i. Similarly, we will have 
fr 1 (y) = /r (y)- By n °ting that T\ is again a localized 
subtree of T\, (10) holds for the case in which pivoting is 
performed as well. The above completes the proof of the 
induction from t = to t = 1. The induction from i-th 
iteration to t + 1-th iteration follows analogously. The proof 
of Theorem 3 is complete. 



Appendix IV 
PROOF OF Theorem 4 

Theorem 4 is the culmination of all analyses of Algorithm 3, 
the proof of which needs the following concepts / notation in 
addition to those established in APPENDICES I through III. 

The Transcribed Tree T 

Following the same notation as in APPENDIX III, let T Xi 
denote the infinite-sized tree corresponding to the iterative 
decoder /i(y). Denote a series of localized finite subtrees of 
T Xi by . . . ,T Xi , s , ■ ■ •} such that T Xi $ contains only 

one variable node Vi and its immediate children. T Xi:S+ \ can 
be iteratively constructed from T XuS by adding a single leaf to 
T XiyS . Note: Unlike Algorithm 3, no pivoting is performed this 
time. T XitS just includes more and more of T Xi as s becomes 
large. 

Let % denote the tree resulting from Algorithm 3 after 
t iterations of the REPEAT-UNTIL loop in Line 3. The 
definition of the transcript tree is as follows. 

Definition 4: For fixed t and s, T t is a tree transcribed from 
%i,s if /r t (y) = fT Xi , s (y) for all possible y. 
Namely, a transcribed tree % is a tree of a different structure 
than % uS but is equivalent to % uS from the output value 
perspective. 

The Balanced Growing LF Module 

A balanced growing leaf finding (LF) module is defined as 
follows. 

Definition 5 (The Balanced Growing LF Module): With 
the assumption that Algorithm 3 is combined with the 
pruning rules in Section V-A, the corresponding LF 
module is balanced growing with respect to the series 
{T Xi fi, . . . , T XitS , . . .} if there exists an increasing list of 
time instants t\ < ti < . . . < t s < . . ., such that % a is a 
transcribed tree of T XitS for all s 6 N. 

Proposition 5: The balanced growing LF module exists for 
any sequence {T Xifi , T XitS , . . .}. 

Proof: This proposition will be proved by explicit 
construction. 

When s = 0, simply set t = 0. Since % a contains only 
variable node Vi and its immediate children and is identical 
to the first entry T Xi $, T to is a transcribed tree of T Xi fi. 
Proposition 5 holds for s = 0. 

Suppose Proposition 5 holds for the general s-th entry, 
i.e., % s is a transcribed tree of T Xi s . Suppose T Xi s+1 is 
obtained from T Xi _ s by adding a leaf v k . Using the notation 
of Appendix III, let T t denote the infinite-sized tree after 
t s iterations, of which T ts a localized subtree. 



We first note that the pivoting operation involves duplicating 
trees, which results in a one-to-many relationship between 
nodes in the original tree and nodes in the pivoted tree. Due 
to the fact that the pivoting operation may be performed many 
times before arriving at % s , there may be more than one leaves 
in T ts but not in T ts that correspond to the to-be-added leaf Vk 
for T s . All these leaves are adjacent to % s and are candidates 
of the next to-be-added leaf for the (t s + l)-th iteration of 
Algorithm 3. We denote these leaves as , , . . . , v^"\ If 
all these leaves can be added to % s successfully, we will obtain 
a T ts+1 that is a transcribed tree of T^.s+i, which completes 
the proof. So the remaining question is how to add all of them 
successfully to T ts . 

For the (t s + l)-th iteration, we design the LF module in 
Line 4 to choose based on which the iteration proceeds. 
If yca(w^) is a check node, no pivoting will be performed and 
Vfp is added to T ts directly without any unexpected side effect. 
(w — 1) leaves . . . ,v k w ^ remain. For the case in which 
yca(w^, 1 - ) ) is a variable node, a pivoting will be performed 
and subtrees will be duplicated. Some nodes in the previous 
remaining list v ^ , . . . , v k w ^ are duplicated as well, and the 
overall effect may be a list , . . . , , v^"^ , . . . , V ( ^ )+Wd ^ 
longer than the original one. We use the last Wd nodes to 
denote the duplicated leaves resulting from pivoting. This node 
duplication effect presents the major challenge of adding all 
i v k} mt0 1~t s - The design goal is to have a LF module that 
does not result in an ever-increasing list of {v^}. 

Without loss of generality, we assume that v k 2 \ . . . , v k 1+Wd ^ 
are the Wd leaves in T ts that will be duplicated by the left 
and the right subtrees in Fig. 4(b). After pivoting, the original 
leaves are duplicated and have identical copies in the left and 
the right subtrees. We use the first Wd leaves . . . , v^ k +Wd ^ 
to represent the corresponding leaves in the right subtree, and 
use v k w+1 \ . . . jV^^^ for the corresponding leaves in the 
left subtree. For the (< s + 2)-th iteration, the LF module locates 
as tne next to-be-added leaf. Since the root of the 
left subtree, denoted as v, is an active variable node with 
free observation yk, yca(w^, u ' +1 ^) is simply v. By invoking 
Rule 4, no pivoting is necessary and we can directly include 
v k w+1 ^ by setting its incoming observation y k = 1. By adding 
the remaining v^ k w+2 \ . . . , ^™' +u ' ti ' into T tg+ i sequentially, 
and noting that only Rule 4 will be invoked, we are able to 
shortened the list of corresponding leaves to . . . ,v^ k w \ 
In sum, the end result of the above construction is that 
regardless of whether yca(w^) is a variable node or not, this 
LF module is able to successfully add into % s without 
introducing any new leaves. Repeating the same procedure 
for . . . , v k w \ we have constructed a LF module that 
generates T tg+1 , a transcribed tree of T x ^ s . The LF module 
under this construction is thus balanced growing. The proof is 
complete. ■ 
Proof of Theorem 4: With the concept of balanced- 
growing LF modules, the proof of Theorem 4 becomes 
straightforward. We construct a series of finite trees 
{T Xifi} . . . ,T Xi ^ s , . . .} by the breadth first search. Its corre- 
sponding balanced growing LF module is then one optimal 
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LF module satisfying Theorem 4, the justification of which is 
as follows. 

Let s denote the first s such that the breadth first search 
has visited all nodes of depth of depth 2n. Since the iterative 
decoder stops after at most n rounds of message exchanging, 
the finite tree T XitSo is equivalent to the infinite tree T l in 
terms of function outputs. The balanced growing LF module 
guarantees that after a finite time to = t So , % Q is a transcribed 
tree of % uSa - Therefore /r t0 (y) = /r X4 ,. (y) = fr Xi (y) = 
fi(y) for all y, which implies that X to is the collection of all 
stopping sets involving x,. By Theorem 3, Vt > to, X t is the 
collection of all stopping sets containing x i7 and the proof is 
complete. ■ 
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